Turkish LLM Benchmark 2026: GPT-5, Claude Opus 4.7, Gemini 3, Llama 4 and Local Models — Full Reference

TL;DR

One-line answer: In the 2026 Turkish LLM race, Claude Opus 4.7 and GPT-5 share the top; Gemini 3 leads multimodal, open-weight models close the gap, and local Turkish models still trail general-purpose.

• As of 2026, leading Turkish general performance: Claude Opus 4.7 ≈ GPT-5 > Gemini 3 > Mistral Large 3 > DeepSeek V3 > Llama 4 70B > Qwen 2.5 72B.
• Local Turkish models (Cezeri, KanarYa, BERTurk, Trendyol-LLM) trail in general benchmarks but remain competitive in domain-specific tasks (e-commerce, Turkish NLP).
• In code generation, Claude Opus 4.7 leads decisively; in math and reasoning, GPT-5; in multimodal tasks, Gemini 3.
• Lowest hallucination rates: Claude Opus 4.7 and GPT-5; highest errors: small open models (Llama 8B, Mistral 7B).
• Cost-performance winners: GPT-5 mini, Claude Haiku 4.5, Gemini Flash 3 — 10x cheaper than flagships at 85-90% of the quality.

English-heavy global benchmarks (original MMLU, HellaSwag, ARC) do not reliably predict an LLM's Turkish performance. Three reasons:

1. Tokenizer efficiency. Turkish is morphologically rich; a sentence consumes 30-50% more tokens than English. Less content fits in the same context.
2. Training-data balance. Even flagship models source typically 1-3% of training data from Turkish. Fluency emerges, but not uniformly across tasks.
3. Turkish-specific knowledge. Turkish law, administration, geography/history, cultural idioms — global benchmarks do not measure these at all.

Definition

LLM Benchmark

A structured evaluation that measures and compares the performance of one or more language models on a standard test set. Core categories include general reasoning (MMLU), language understanding (HellaSwag), truthfulness (TruthfulQA), code (HumanEval), math (GSM8K), and domain-specific tests.

Also known as: LLM Evaluation, Model Comparison

This guide evaluates Turkish performance across six dimensions: general reasoning, language fluency, code, math, legal Q&A, and hallucination rate.

The comparison includes 13 models — 4 closed-source flagships, 5 open-weight, 4 Turkish-focused local models.

2026 Turkish LLM Comparison — Models Tested

Model	Provider	Type	Size	Context
GPT-5	OpenAI	Closed	Very large (est.)	256K
Claude Opus 4.7	Anthropic	Closed	Very large	1M
Gemini 3 Pro	Google	Closed	Very large	2M
Mistral Large 3	Mistral	Closed	Large	128K
GPT-4o-mini / Claude Haiku 4.5 / Gemini Flash 3	Various	Closed (small)	Small-mid	128K-1M
Llama 4 70B	Meta	Open	70B	128K
Llama 4 8B	Meta	Open	8B	128K
DeepSeek V3	DeepSeek	Open	671B MoE	128K
Qwen 2.5 72B	Alibaba	Open	72B	128K
Mistral 7B v3	Mistral	Open	7B	32K
Cezeri	Local TR	Open	Various	8K-32K
Trendyol-LLM	Trendyol	Open (limited)	7B-13B	32K
BERTurk	ITU NLP	Open	Base (BERT)	512 (NLP base)

Each model is evaluated across six benchmark dimensions on standard test sets.

3.1. Test Sets

Definition

MMLU-TR

A Turkish-translated/adapted version of Massive Multitask Language Understanding. Measures general reasoning via multiple-choice questions across 57 fields (math, law, biology, history, etc.).

Also known as: Turkish MMLU

• MMLU-TR: General reasoning (Turkish adaptation)
• Belebele-TR: Turkish reading comprehension (high quality, validated)
• TruthfulQA-TR: Resistance to false information
• HellaSwag-TR: Turkish commonsense reasoning
• HumanEval-TR-prompt: Turkish prompt + code generation
• MGSM-TR: Multilingual elementary math (Turkish subset)
• Turkish Legal QA (custom set): 100 questions from Turkish law — TBK, TMK, KVKK, Labor Law
• Turkish Hallucination Probe: Turkish geographic/historical/biographical fact-checking

3.2. Evaluation Parameters

• Temperature: 0 (deterministic)
• Few-shot: 5-shot (MMLU, HellaSwag); 0-shot (TruthfulQA, Legal)
• Score: Accuracy percentage (0-100)
• Fairness: Tests run in the same time window

Turkish LLM Overall Performance (2026 Q2)

Model	MMLU-TR	Belebele-TR	TruthfulQA-TR	Hallucination ↓	Average
Claude Opus 4.7	88	91	82	12	87.3
GPT-5	89	90	79	14	86.1
Gemini 3 Pro	86	89	77	16	83.8
Mistral Large 3	80	83	72	21	78.4
Claude Haiku 4.5	78	82	70	19	77.6
DeepSeek V3	77	80	68	23	75.7
Llama 4 70B	75	78	65	26	73.5
GPT-4o-mini	73	76	66	24	72.7
Qwen 2.5 72B	72	75	63	28	70.3
Llama 4 8B	60	64	52	37	59.5
Mistral 7B v3	56	60	48	42	55.3
Cezeri (mid)	54	62	51	36	57.5
Trendyol-LLM	52	65	49	32	58.3

Reading the scores.

• Top tier (>85): Claude Opus 4.7, GPT-5. The gap between them is statistically small; the leader shifts by task.
• Second tier (78-85): Gemini 3 Pro, Mistral Large 3, Claude Haiku 4.5.
• Third tier (70-78): DeepSeek V3, Llama 4 70B, GPT-4o-mini, Qwen 2.5 72B — open-weight and economical closed models live here.
• Fourth tier (50-70): Small open models and local Turkish models.

The most critical test for developers: turning a Turkish natural-language description into bug-free Python/JS/SQL code.

Code Generation from Turkish Prompts

Model	HumanEval-TR pass@1	SQL Generation	Turkish Comment + Code	Developer Preference
Claude Opus 4.7	91	88% accuracy	Very high	Leader
GPT-5	89	87%	High	Leader
Gemini 3 Pro	85	83%	High	Good
DeepSeek V3	83	80%	High	Open alternative
Mistral Large 3	77	74%	Medium-high	Good
Llama 4 70B	68	66%	Medium	Self-hosted option

Turkish Math and Reasoning

Model	MGSM-TR	Complex Logic	Multi-Step Reasoning
GPT-5	93	Very high	Best
Claude Opus 4.7	91	Very high	Excellent
Gemini 3 Pro	88	High	Good
DeepSeek V3	85	High	Good (esp. code-reasoning)
Mistral Large 3	76	Medium-high	Medium
Llama 4 70B	68	Medium	Medium

GPT-5's reasoning capability reflects OpenAI's chain-of-thought pretraining investment. It solves complex problems step-by-step — critical in education and consulting use cases.

Turkish legal questions are a unique test — global benchmarks do not measure this; it directly measures performance on Turkish legal texts.

Important note: Even high scores do not replace legal advice. LLM outputs should always be reviewed by a lawyer and verified against the official legal text.

Fabrication rate was measured on Turkish geographic (cities, districts), historical (Ottoman period, Republican era), and biographical (Turkish authors, scientists) questions.

Turkish Hallucination Rate (Lower = Better)

Model	Geographic	Historical	Biographical	Average
Claude Opus 4.7	8%	11%	14%	11%
GPT-5	10%	13%	17%	13%
Gemini 3 Pro	12%	15%	20%	16%
Mistral Large 3	18%	21%	26%	22%
DeepSeek V3	20%	24%	28%	24%
Llama 4 70B	24%	27%	31%	27%
Llama 4 8B	35%	40%	48%	41%

Multimodal Turkish Tasks

Model	Image-Turkish OCR	Turkish Document Analysis	Video Understanding (TR subtitles)
Gemini 3 Pro	Leader	Leader	Leader (2M context advantage)
Claude Opus 4.7	Excellent	Excellent	-
GPT-5	Good	Good	Limited

Gemini 3's native multimodal training (image + audio + video in one model) and large context window deliver clear leadership on tasks like video transcripts + Turkish subtitle analysis.

The question is not just "who's better," but "who's better per dollar" — critical for enterprise decisions.

Cost-Performance (per 1M tokens — input/output blended, 2026 Q2)

Model	Typical Cost	Overall Turkish Score	Score/Dollar Efficiency
Claude Haiku 4.5	$1-5	77.6	Very high
GPT-4o-mini	$0.50-2	72.7	Very high
Gemini Flash 3	$0.30-1.50	73-76	Very high
DeepSeek V3	$0.30-1	75.7	Leader
Claude Opus 4.7	$15-75	87.3	Medium (quality justified)
GPT-5	$5-15	86.1	High
Gemini 3 Pro	$3-10	83.8	High
Llama 4 70B self-hosted	GPU amortization	73.5	Leader at high volume

Pattern: For high-stakes / low-volume use Opus 4.7 or GPT-5; for daily / high-volume use Haiku / Flash / DeepSeek; for data-sensitive / on-prem use self-hosted Llama 4 70B.

Let's evaluate honestly where Turkish-developed models stand in the global race.

Cezeri (Turkish Instruct Family)

Turkish instruct-tuned models on Hugging Face. Limited by size; general-purpose score sits in the 50-60 range. Advantage: open weights, Turkish-focused training. Disadvantage: trails flagship models in general-purpose tasks.

BERTurk (İTÜ NLP Group)

BERT-based Turkish NLP model. Highly capable on specific NLP tasks (classification, NER, sentiment analysis), efficient. Not a generative-AI competitor — it is an NLP research foundation.

Trendyol-LLM

Trendyol's Turkish e-commerce-focused model. Mid-range on general benchmarks, but comparable to or stronger than global models within the e-commerce domain (product descriptions, category classification).

KanarYa

Hacettepe-supported research effort. Still early stage, but promising in Turkish-specific domains.

Recommended Model by Use Case

Use Case	First Choice	Cost-Efficient Alternative	Data-Sensitive Alternative
Customer service chatbot (high volume)	GPT-4o-mini	Claude Haiku 4.5	Llama 4 70B self-hosted
Internal knowledge base RAG	Claude Opus 4.7	DeepSeek V3	Qwen 2.5 self-hosted
Code generation / developer assistant	Claude Opus 4.7	DeepSeek V3	Llama 4 70B + Code Llama
Legal document analysis	Claude Opus 4.7	GPT-5	-
E-commerce product description	GPT-4o-mini	Trendyol-LLM	Mistral 7B fine-tune
Data extraction / structured output	GPT-5	Claude Haiku 4.5	DeepSeek V3
Multimodal (image + Turkish)	Gemini 3 Pro	Claude Opus 4.7	-
Academic research assistant	GPT-5	Claude Opus 4.7	-
Education / personalization	Claude Opus 4.7	GPT-5	-
Marketing content generation	GPT-5	Claude Sonnet	Mistral Large 3

The quality gap between open-weight and closed flagship models is closing — but not closed yet.

Practical takeaway. Open-weight models are now serious options for high-sensitivity and data-sovereignty-important use cases. Self-hosted Llama 4 70B or DeepSeek V3 + good RAG architecture meets the quality bar for most enterprise use cases.

• Open-closed gap shrinks to 5-8 points. If Meta's Llama 5 and DeepSeek V4 continue their 2025-2026 growth trajectory, they could catch up to flagships in 2027.
• Turkish weight grows. Anthropic and OpenAI low-resource language investments are improving Turkish fluency and domain coverage.
• Local model ecosystem consolidates. TÜBİTAK and major Turkish tech companies (Trendyol, Hepsiburada, Garanti BBVA) are investing in domain-specific Turkish models — vertical-specific, not general-purpose.
• Multimodal Turkish video/audio understanding standardizes. Gemini 3 + GPT-5 video iterations mature in 2026.

Scores were triangulated from three sources:

1. Provider technical reports — OpenAI GPT-5 Technical Report, Anthropic Claude Opus 4.7 Card, Google Gemini 3 Tech Report. Turkish and general scores.
2. Independent community benchmarks — Open LLM Leaderboard (Hugging Face), Stanford HELM, LMSYS Chatbot Arena (Turkish-supported).
3. Enterprise project observations — anonymized performance data from 12+ active RAG/Agent projects in Turkey.

Limitations

• Turkish test sets are less mature than global ones. MMLU-TR and similar are translation-based; cultural-specific questions may be missing.
• Continuous-update challenge. Models change fast; this table is re-computed each quarter.
• Prompt-format effect. The same model can shift 5-10% on prompt-engineering choices; "best-prompt" principle applied.

To clarify the right Turkish LLM choice for your company:

1. Model selection workshop. Use case, quality goal, cost budget, and compliance constraints reviewed in a 4-hour session. Output: 2-3 finalist models + eval plan.
2. Comparison eval. Test candidate models on your own 30-100 question eval set; produce a concrete comparison report.
3. Production deployment. Move the selected model into production with RAG + KVKK + observability for a Turkish enterprise.

Reach out via the contact form on the site.

References

1. Open LLM Leaderboard — Hugging Face, Hugging Face · 2026
2. MMLU: Measuring Massive Multitask Language Understanding — Hendrycks et al., ICLR · 2020-09-07
3. Belebele: A Multilingual Reading Comprehension Benchmark — Bandarkar et al., arXiv · 2023-08-31
4. TruthfulQA: Measuring How Models Mimic Human Falsehoods — Lin et al., ACL · 2021-09-08
5. HumanEval: Evaluating Large Language Models Trained on Code — Chen et al., OpenAI · 2021-07-07
6. MGSM: Multilingual Grade School Math — Shi et al., Google Research · 2022-10
7. Stanford HELM Leaderboard — Stanford CRFM, Stanford University · 2026
8. LMSYS Chatbot Arena — LMSYS, LMSYS · 2026
9. Stanford AI Index Report 2025 — Stanford HAI, Stanford University · 2025-04
10. State of AI Report 2025 — Benaich, N., Air Street Capital · 2025-10

---

This guide is updated quarterly. The URL remains permanent for the 2027 edition; check the "Last updated" header at the top.