Skip to content
Weights Not Walls
Menu

DTBench

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://conceptualreasoning.ai/dtbench
Score shown
Epoch's column "Accuracy", as a percentage, higher is better; random guessing scores about 40.0%. The source gives no error bars.
Results
241 in all: 75 for open-weight models, 161 for closed ones, 5 where Epoch does not say. 59 linked to a model page here.

Best open-weight result: deepseek-ai/DeepSeek-V4-Pro-0813, 93.9%, number 66 of 241 results.

DTBench results
#ModelWeightsSettingScoreDateSource
1claude-fable-5-2_unknownAnthropicclosedeffort unknown99.2%not datedconceptualreasoning.ai
2claude-fable-5-2_xhighAnthropicclosedeffort xhigh98.9%not datedconceptualreasoning.ai
3claude-opus-5-5_maxAnthropicclosedeffort max98.9%not datedconceptualreasoning.ai
4claude-fable-5_maxAnthropicclosedeffort max98.4%not datedconceptualreasoning.ai
5claude-fable-5-2_highAnthropicclosedeffort high98.1%not datedconceptualreasoning.ai
6claude-fable-5-2_lowAnthropicclosedeffort low98.1%not datedconceptualreasoning.ai
7claude-fable-5-2_mediumAnthropicclosedeffort medium98.1%not datedconceptualreasoning.ai
8claude-opus-5_highAnthropicclosedeffort high97.9%not datedconceptualreasoning.ai
9claude-opus-5_xhighAnthropicclosedeffort xhigh97.9%not datedconceptualreasoning.ai
10claude-fable-5-1_unknownAnthropicclosedeffort unknown97.6%not datedconceptualreasoning.ai
11claude-fable-5-1_xhighAnthropicclosedeffort xhigh97.6%not datedconceptualreasoning.ai
12claude-opus-5_maxAnthropicclosedeffort max97.6%not datedconceptualreasoning.ai
13gemini-4-argon_lowGoogle DeepMindclosedeffort low97.6%not datedconceptualreasoning.ai
14claude-fable-5_xhighAnthropicclosedeffort xhigh97.3%not datedconceptualreasoning.ai
15claude-opus-5-5_xhighAnthropicclosedeffort xhigh97.3%not datedconceptualreasoning.ai
16claude-sonnet-5-5_maxAnthropicclosedeffort max97.3%not datedconceptualreasoning.ai
17gemini-4-argon_highGoogle DeepMindclosedeffort high97.3%not datedconceptualreasoning.ai
18gemini-4-argon_mediumGoogle DeepMindclosedeffort medium97.3%not datedconceptualreasoning.ai
19gpt-6-astra_xhighOpenAIclosedeffort xhigh97.3%not datedconceptualreasoning.ai
20gpt-6-sol_maxOpenAIclosedeffort max97.3%not datedconceptualreasoning.ai
21grok-4.6_xhighxAIclosedeffort xhigh97.3%not datedconceptualreasoning.ai
22claude-fable-5-1_mediumAnthropicclosedeffort medium97.1%not datedconceptualreasoning.ai
23gemini-3.1-pro-preview_mediumGoogle DeepMindclosedeffort medium97.1%not datedconceptualreasoning.ai
24gpt-6-astra_mediumOpenAIclosedeffort medium97.1%not datedconceptualreasoning.ai
25gpt-6-astra_unknownOpenAIclosedeffort unknown97.1%not datedconceptualreasoning.ai
26gemini-3.7-flash_highGoogle DeepMindclosedeffort high96.8%not datedconceptualreasoning.ai
27claude-fable-5_highAnthropicclosedeffort high96.5%not datedconceptualreasoning.ai
28claude-fable-5-1_highAnthropicclosedeffort high96.5%not datedconceptualreasoning.ai
29claude-opus-5-5_highAnthropicclosedeffort high96.5%not datedconceptualreasoning.ai
30claude-opus-5-5_mediumAnthropicclosedeffort medium96.5%not datedconceptualreasoning.ai
31gpt-6-astra_highOpenAIclosedeffort high96.5%not datedconceptualreasoning.ai
32gpt-6.1-sol_maxOpenAIclosedeffort max96.5%not datedconceptualreasoning.ai
33grok-4.5_highxAIclosedeffort high96.5%not datedconceptualreasoning.ai
34muse-spark-1.3_unknownMeta AIclosedeffort unknown96.5%not datedconceptualreasoning.ai
35claude-sonnet-5-5_xhighAnthropicclosedeffort xhigh96.0%not datedconceptualreasoning.ai
36gpt-5.5_xhighOpenAIclosedeffort xhigh96.0%not datedconceptualreasoning.ai
37gpt-5.5-pro_xhighOpenAIclosedeffort xhigh96.0%not datedconceptualreasoning.ai
38gpt-5.6-sol_promaxOpenAIcloseddefault96.0%not datedconceptualreasoning.ai
39grok-4.7_xhighxAIclosedeffort xhigh96.0%not datedconceptualreasoning.ai
40claude-fable-5_mediumAnthropicclosedeffort medium95.7%not datedconceptualreasoning.ai
41gpt-5.6-sol_xhighOpenAIclosedeffort xhigh95.7%not datedconceptualreasoning.ai
42gemini-3.8-flash_unknownGoogle DeepMindclosedeffort unknown95.7%not datedconceptualreasoning.ai
43claude-opus-5-5_lowAnthropicclosedeffort low95.5%not datedconceptualreasoning.ai
44gemini-3.1-pro-preview_highGoogle DeepMindclosedeffort high95.5%not datedconceptualreasoning.ai
45gemini-3.6-flash_highGoogle DeepMindclosedeffort high95.5%not datedconceptualreasoning.ai
46gpt-5.6-sol_highOpenAIclosedeffort high95.5%not datedconceptualreasoning.ai
47gpt-5.6-sol_maxOpenAIclosedeffort max95.5%not datedconceptualreasoning.ai
48gpt-5.6-sol_mediumOpenAIclosedeffort medium95.2%not datedconceptualreasoning.ai
49muse-wonder-1.0_unknownMeta AIclosedeffort unknown95.2%not datedconceptualreasoning.ai
50claude-haiku-5-5_maxAnthropicclosedeffort max95.0%not datedconceptualreasoning.ai
51claude-opus-4-8_maxAnthropicclosedeffort max94.9%not datedconceptualreasoning.ai
52claude-opus-5_mediumAnthropicclosedeffort medium94.9%not datedconceptualreasoning.ai
53gemini-3.1-pro-preview_lowGoogle DeepMindclosedeffort low94.9%not datedconceptualreasoning.ai
54gemini-3.8-flash_lowGoogle DeepMindclosedeffort low94.9%not datedconceptualreasoning.ai
55gemini-3.8-flash_mediumGoogle DeepMindclosedeffort medium94.9%not datedconceptualreasoning.ai
56gpt-6-astra_lowOpenAIclosedeffort low94.9%not datedconceptualreasoning.ai
57gemini-3.5-flash_highGoogle DeepMindclosedeffort high94.7%not datedconceptualreasoning.ai
58gemini-3.7-flash_mediumGoogle DeepMindclosedeffort medium94.7%not datedconceptualreasoning.ai
59muse-spark-1.2_xhighMeta AIclosedeffort xhigh94.7%not datedconceptualreasoning.ai
60claude-opus-4-7_maxAnthropicclosedeffort max94.7%not datedconceptualreasoning.ai
61gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh94.4%not datedconceptualreasoning.ai
62muse-spark-1.1_highMeta AIclosedeffort high94.4%not datedconceptualreasoning.ai
63claude-fable-5-1_lowAnthropicclosedeffort low94.1%not datedconceptualreasoning.ai
64claude-sonnet-5-5_highAnthropicclosedeffort high94.1%not datedconceptualreasoning.ai
65claude-sonnet-5-5_mediumAnthropicclosedeffort medium93.9%not datedconceptualreasoning.ai
66deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_unknown, DeepSeekopeneffort unknown93.9%not datedconceptualreasoning.ai
67gpt-5.6-sol_lowOpenAIclosedeffort low93.9%not datedconceptualreasoning.ai
68zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI)openeffort max93.6%not datedconceptualreasoning.ai
69claude-sonnet-5-5_lowAnthropicclosedeffort low93.3%not datedconceptualreasoning.ai
70gemini-3.7-flash_lowGoogle DeepMindclosedeffort low93.3%not datedconceptualreasoning.ai
71gpt-5.6-terra_maxOpenAIclosedeffort max93.3%not datedconceptualreasoning.ai
72moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro_max, Moonshotopeneffort max93.3%not datedconceptualreasoning.ai
73claude-opus-5_lowAnthropicclosedeffort low93.1%not datedconceptualreasoning.ai
74gpt-5.6-terra_xhighOpenAIclosedeffort xhigh92.5%not datedconceptualreasoning.ai
75claude-sonnet-5_maxAnthropicclosedeffort max92.5%not datedconceptualreasoning.ai
76qwen3.7-maxAlibabacloseddefault92.3%not datedconceptualreasoning.ai
77qwen3.8-max_xhighAlibabaclosedeffort xhigh92.0%not datedconceptualreasoning.ai
78claude-fable-5_lowAnthropicclosedeffort low91.5%not datedconceptualreasoning.ai
79claude-opus-4-6_maxAnthropicclosedeffort max91.2%not datedconceptualreasoning.ai
80gpt-5.6-terra_highOpenAIclosedeffort high91.2%not datedconceptualreasoning.ai
81moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshotopeneffort max91.2%not datedconceptualreasoning.ai
82deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeekopeneffort max90.9%not datedconceptualreasoning.ai
83gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh90.9%not datedconceptualreasoning.ai
84moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault90.9%not datedconceptualreasoning.ai
85deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeekopeneffort max90.7%not datedconceptualreasoning.ai
86gpt-5-2025-08-07_highOpenAIclosedeffort high90.7%not datedconceptualreasoning.ai
87grok-4.3_highxAIclosedeffort high90.7%not datedconceptualreasoning.ai
88gpt-5.1-2025-11-13_highOpenAIclosedeffort high90.1%not datedconceptualreasoning.ai
89gpt-5.6-terra_lowOpenAIclosedeffort low90.1%not datedconceptualreasoning.ai
90gpt-5.6-terra_mediumOpenAIclosedeffort medium90.1%not datedconceptualreasoning.ai
91gpt-6-luna_maxOpenAIclosedeffort max90.1%not datedconceptualreasoning.ai
92grok-4-20xAIcloseddefault90.1%not datedconceptualreasoning.ai
93nemotron-3-ultraNvidiano page matchopendefault90.1%not datedconceptualreasoning.ai
94claude-opus-4-5-20251101_highAnthropicclosedeffort high89.9%not datedconceptualreasoning.ai
95claude-sonnet-4-6_maxAnthropicclosedeffort max89.9%not datedconceptualreasoning.ai
96deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_unknown, DeepSeekopeneffort unknown89.9%not datedconceptualreasoning.ai
97claude-sonnet-5_xhighAnthropicclosedeffort xhigh89.5%not datedconceptualreasoning.ai
98gemini-3-flash-preview_highGoogle DeepMindclosedeffort high89.1%not datedconceptualreasoning.ai
99gpt-5.6-luna_xhighOpenAIclosedeffort xhigh89.1%not datedconceptualreasoning.ai
100gpt-5.6-luna_maxOpenAIclosedeffort max88.8%not datedconceptualreasoning.ai
101mistral-large-4-0_unknownMistral AIno page matchopeneffort unknown88.2%not datedconceptualreasoning.ai
102Qwen/Qwen3.8-27BEpoch: qwen3.8-27b, Alibabaopendefault88.0%not datedconceptualreasoning.ai
103deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_thinking, DeepSeekopeneffort thinking87.7%not datedconceptualreasoning.ai
104grok-4-1-fast-reasoningxAIcloseddefault87.7%not datedconceptualreasoning.ai
105zai-org/GLM-5.3Epoch: glm-5.3_unknown, Z.ai (Zhipu AI)openeffort unknown87.7%not datedconceptualreasoning.ai
106thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machinesopeneffort xhigh87.5%not datedconceptualreasoning.ai
107Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibabaopendefault87.5%not datedconceptualreasoning.ai
108claude-haiku-5-5_xhighAnthropicclosedeffort xhigh87.2%not datedconceptualreasoning.ai
109qwen3.6-max-previewAlibabanot stateddefault87.2%not datedconceptualreasoning.ai
110o3-pro-2025-06-10_highOpenAIclosedeffort high86.9%not datedconceptualreasoning.ai
111claude-haiku-5-5_highAnthropicclosedeffort high86.7%not datedconceptualreasoning.ai
112gpt-5.6-luna_highOpenAIclosedeffort high86.7%not datedconceptualreasoning.ai
113deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_max, DeepSeekopeneffort max86.4%not datedconceptualreasoning.ai
114deepseek-ai/DeepSeek-V3.2Epoch: deepseek/deepseek-v3.2, DeepSeekopendefault85.6%not datedconceptualreasoning.ai
115o3-2025-04-16_highOpenAIclosedeffort high84.8%not datedconceptualreasoning.ai
116claude-sonnet-5_highAnthropicclosedeffort high84.5%not datedconceptualreasoning.ai
117XiaomiMiMo/MiMo-V2.5-ProEpoch: mimo-v2.5-pro, Xiaomi Corpopendefault84.5%not datedconceptualreasoning.ai
118claude-haiku-5-5_mediumAnthropicclosedeffort medium84.3%not datedconceptualreasoning.ai
119Qwen/Qwen3.5-122B-A10BEpoch: qwen3.5-122b-a10b_none, Alibabaopeneffort none84.3%not datedconceptualreasoning.ai
120qwen3.7-plusAlibabacloseddefault84.0%not datedconceptualreasoning.ai
121gpt-5.6-sol_noneOpenAIclosedeffort none83.7%not datedconceptualreasoning.ai
122gemini-3.5-flash-lite_highGoogle DeepMindclosedeffort high83.5%not datedconceptualreasoning.ai
123claude-sonnet-4-5-20250929_unknownAnthropicclosedeffort unknown83.2%not datedconceptualreasoning.ai
124gpt-5.6-luna_mediumOpenAIclosedeffort medium83.2%not datedconceptualreasoning.ai
125qwen3.5-flashAlibabanot stateddefault82.9%not datedconceptualreasoning.ai
126deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1_thinking, DeepSeekopeneffort thinking82.7%not datedconceptualreasoning.ai
127google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMindopendefault82.7%not datedconceptualreasoning.ai
128grok-4-fastxAIcloseddefault82.7%not datedconceptualreasoning.ai
129gemini-2.5-proGoogle DeepMindcloseddefault82.4%not datedconceptualreasoning.ai
130Qwen/Qwen3.5-27BEpoch: qwen3.5-27B, Alibabaopendefault82.4%not datedconceptualreasoning.ai
131qwen3-max-2025-09-23Alibabacloseddefault82.1%not datedconceptualreasoning.ai
132qwen3.6-plusAlibabacloseddefault81.9%not datedconceptualreasoning.ai
133claude-opus-4-20250514_unknownAnthropicclosedeffort unknown81.6%not datedconceptualreasoning.ai
134deepseek-ai/DeepSeek-V3.1-TerminusEpoch: DeepSeek-V3.1-Terminus, DeepSeekopendefault81.3%not datedconceptualreasoning.ai
135claude-haiku-5-5_lowAnthropicclosedeffort low81.1%not datedconceptualreasoning.ai
136qwen-plus-2025-04-28Alibabacloseddefault81.1%not datedconceptualreasoning.ai
137gpt-5-mini-2025-08-07_highOpenAIclosedeffort high80.5%not datedconceptualreasoning.ai
138qwen3.5-plusAlibabanot stateddefault80.5%not datedconceptualreasoning.ai
139gpt-5.4-nano-2026-03-17_xhighOpenAIclosedeffort xhigh80.3%not datedconceptualreasoning.ai
140Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault80.3%not datedconceptualreasoning.ai
141claude-opus-4-1-20250805_unknownAnthropicclosedeffort unknown80.0%not datedconceptualreasoning.ai
142gpt-5.4-mini-2026-03-17_xhighOpenAIclosedeffort xhigh80.0%not datedconceptualreasoning.ai
143gpt-5.6-luna_lowOpenAIclosedeffort low80.0%not datedconceptualreasoning.ai
144Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibabaopendefault80.0%not datedconceptualreasoning.ai
145claude-sonnet-5_mediumAnthropicclosedeffort medium79.2%not datedconceptualreasoning.ai
146MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMaxopendefault78.9%not datedconceptualreasoning.ai
147gpt-5.6-terra_noneOpenAIclosedeffort none78.7%not datedconceptualreasoning.ai
148Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: Qwen3-235B-A22B-Instruct-2507, Alibabaopendefault78.4%not datedconceptualreasoning.ai
149Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibabaopendefault78.1%not datedconceptualreasoning.ai
150o4-mini-2025-04-16_highOpenAIclosedeffort high77.6%not datedconceptualreasoning.ai
151claude-sonnet-4-20250514_unknownAnthropicclosedeffort unknown77.1%not datedconceptualreasoning.ai
152qwen3.6-flashAlibabanot stateddefault77.1%not datedconceptualreasoning.ai
153gemini-3.1-flash-lite_highGoogleclosedeffort high76.8%not datedconceptualreasoning.ai
154gemini-2.5-flashGoogle DeepMindcloseddefault76.5%not datedconceptualreasoning.ai
155openai/gpt-oss-120bEpoch: gpt-oss-120b_high, OpenAIopeneffort high76.3%not datedconceptualreasoning.ai
156Qwen/Qwen3-235B-A22BEpoch: Qwen3-235B-A22B, Alibabaopendefault75.7%not datedconceptualreasoning.ai
157mistral-medium-2604Mistral AIno page matchopendefault75.5%not datedconceptualreasoning.ai
158claude-haiku-5-5_noneAnthropicclosedeffort none75.2%not datedconceptualreasoning.ai
159gemma-4-26b-a4bGoogle DeepMindno page matchopendefault74.9%not datedconceptualreasoning.ai
160o1-2024-12-17_highOpenAIclosedeffort high74.7%not datedconceptualreasoning.ai
161Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibabaopendefault73.9%not datedconceptualreasoning.ai
162claude-haiku-4-5-20251001_unknownAnthropicclosedeffort unknown73.6%not datedconceptualreasoning.ai
163claude-sonnet-5_lowAnthropicclosedeffort low73.6%not datedconceptualreasoning.ai
164Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibabaopendefault71.2%not datedconceptualreasoning.ai
165mistral-small-2603Mistral AIno page matchopendefault70.9%not datedconceptualreasoning.ai
166Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibabaopendefault69.3%not datedconceptualreasoning.ai
167gpt-5.6-luna_noneOpenAIclosedeffort none69.3%not datedconceptualreasoning.ai
168gpt-4.1-mini-2025-04-14OpenAIcloseddefault68.8%not datedconceptualreasoning.ai
169o3-mini-2025-01-31_highOpenAIclosedeffort high68.8%not datedconceptualreasoning.ai
170gpt-4.1-2025-04-14OpenAIcloseddefault68.3%not datedconceptualreasoning.ai
171openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAIopeneffort high68.0%not datedconceptualreasoning.ai
172claude-3-5-sonnet-20241022Anthropiccloseddefault67.8%not datedconceptualreasoning.ai
173claude-3-5-sonnet-20240620Anthropiccloseddefault67.8%not datedconceptualreasoning.ai
174mistral-medium-2508Mistral AIcloseddefault67.5%not datedconceptualreasoning.ai
175Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibabaopendefault67.5%not datedconceptualreasoning.ai
176Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibabaopendefault67.2%not datedconceptualreasoning.ai
177grok-2-1212xAIcloseddefault65.2%not datedconceptualreasoning.ai
178mistral-large-2512Mistral AIno page matchopendefault65.1%not datedconceptualreasoning.ai
179deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeekopendefault64.8%not datedconceptualreasoning.ai
180gpt-4o-2024-05-13OpenAIcloseddefault64.5%not datedconceptualreasoning.ai
181Qwen/Qwen3-14BEpoch: qwen3-14b, Alibabaopendefault64.0%not datedconceptualreasoning.ai
182gemini-2.0-flash-001Google DeepMind,Googlecloseddefault63.2%not datedconceptualreasoning.ai
183Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibabaopendefault62.9%not datedconceptualreasoning.ai
184gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMindcloseddefault62.8%not datedconceptualreasoning.ai
185deepseek-chatDeepSeekno page matchopendefault62.7%not datedconceptualreasoning.ai
186gpt-4-0613OpenAIcloseddefault62.7%not datedconceptualreasoning.ai
187gpt-5-nano-2025-08-07_highOpenAIclosedeffort high62.7%not datedconceptualreasoning.ai
188mistral-medium-2505Mistral AIcloseddefault62.3%not datedconceptualreasoning.ai
189meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AIopendefault61.9%not datedconceptualreasoning.ai
190claude-3-opus-20240229Anthropiccloseddefault61.6%not datedconceptualreasoning.ai
191gpt-4-turboOpenAIcloseddefault61.6%not datedconceptualreasoning.ai
192meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AIopendefault61.4%not datedconceptualreasoning.ai
193CohereLabs/c4ai-command-a-03-2025Epoch: c4ai-command-a-03-2025, Cohereopendefault61.3%not datedconceptualreasoning.ai
194mistralai/Magistral-Small-2509Epoch: magistral-small-2509, Mistral AIopendefault61.3%not datedconceptualreasoning.ai
195mistral-large-2407Mistral AIno page matchopendefault61.2%not datedconceptualreasoning.ai
196mistral-large-2411Mistral AIno page matchopendefault60.8%not datedconceptualreasoning.ai
197Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibabaopendefault60.3%not datedconceptualreasoning.ai
198meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AIopendefault60.0%not datedconceptualreasoning.ai
199mistral-small-2506Mistral AIno page matchopendefault59.9%not datedconceptualreasoning.ai
200Qwen/Qwen3-8BEpoch: qwen3-8b, Alibabaopendefault59.7%not datedconceptualreasoning.ai
201meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopendefault59.5%not datedconceptualreasoning.ai
202gemini-1.5-pro-002Google DeepMindcloseddefault59.0%not datedconceptualreasoning.ai
203mistral-small-2503Mistral AIno page matchopendefault58.6%not datedconceptualreasoning.ai
204meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AIopendefault57.9%not datedconceptualreasoning.ai
205claude-3-5-haiku-20241022Anthropiccloseddefault56.7%not datedconceptualreasoning.ai
206gemini-1.5-pro-001Google DeepMindcloseddefault56.5%not datedconceptualreasoning.ai
207mistral-large-2402Mistral AIcloseddefault55.9%not datedconceptualreasoning.ai
208mistralai/Mixtral-8x22B-Instruct-v0.1Epoch: Mixtral-8x22B-Instruct-v0.1, Mistral AIopendefault55.1%not datedconceptualreasoning.ai
209CohereLabs/c4ai-command-r-plus-08-2024Epoch: c4ai-command-r-plus-08-2024, Cohere,Cohere Labs (formerly Cohere for AI)opendefault54.9%not datedconceptualreasoning.ai
210gpt-4o-mini-2024-07-18OpenAIcloseddefault54.4%not datedconceptualreasoning.ai
211meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AIopendefault54.2%not datedconceptualreasoning.ai
212gemini-1.5-flash-002Google DeepMindcloseddefault53.8%not datedconceptualreasoning.ai
213claude-3-sonnet-20240229Anthropiccloseddefault53.6%not datedconceptualreasoning.ai
214mistral-medium-2312Mistral AIcloseddefault53.3%not datedconceptualreasoning.ai
215mistral-small-2402not stateddefault53.0%not datedconceptualreasoning.ai
216gemini-2.0-flash-liteGoogle DeepMindcloseddefault52.5%not datedconceptualreasoning.ai
217google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMindopendefault52.5%not datedconceptualreasoning.ai
218gpt-4.1-nano-2025-04-14OpenAIcloseddefault52.5%not datedconceptualreasoning.ai
219claude-2.0Anthropiccloseddefault51.8%not datedconceptualreasoning.ai
220ministral-3b-2410Mistral AIcloseddefault51.7%not datedconceptualreasoning.ai
221gemini-1.5-flash-001Google DeepMindcloseddefault51.4%not datedconceptualreasoning.ai
222claude-2.1Anthropiccloseddefault50.9%not datedconceptualreasoning.ai
223google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMindopendefault50.9%not datedconceptualreasoning.ai
224meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopendefault50.9%not datedconceptualreasoning.ai
225claude-3-haiku-20240307Anthropiccloseddefault50.1%not datedconceptualreasoning.ai
226gemini-1.5-flash-8b-001Google DeepMindcloseddefault50.0%not datedconceptualreasoning.ai
227open-mixtral-8x7bMistral AIno page matchopendefault49.6%not datedconceptualreasoning.ai
228mistral-small-2409Mistral AIno page matchopendefault49.5%not datedconceptualreasoning.ai
229google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMindopendefault48.8%not datedconceptualreasoning.ai
230mistralai/Mistral-Nemo-Instruct-2407Epoch: Mistral-Nemo-Instruct-2407, Mistral AIopendefault48.6%not datedconceptualreasoning.ai
231gpt-3.5-turbo-0125OpenAIcloseddefault48.5%not datedconceptualreasoning.ai
232gemma-2-27bGoogle DeepMindno page matchopendefault48.0%not datedconceptualreasoning.ai
233Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibabaopendefault47.7%not datedconceptualreasoning.ai
234CohereLabs/c4ai-command-r-08-2024Epoch: c4ai-command-r-08-2024opendefault46.4%not datedconceptualreasoning.ai
235gemini-1.0-pro-001Google DeepMindcloseddefault45.9%not datedconceptualreasoning.ai
236claude-instant-1.2Anthropiccloseddefault45.8%not datedconceptualreasoning.ai
237ministral-8b-2410Mistral AIno page matchopendefault45.7%not datedconceptualreasoning.ai
238meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AIopendefault43.9%not datedconceptualreasoning.ai
239open-mistral-7bMistral AIno page matchopendefault42.5%not datedconceptualreasoning.ai
240meta-llama/Llama-2-13b-chatEpoch: Llama-2-13b-chat, Meta AIopendefault42.2%not datedconceptualreasoning.ai
241meta-llama/Llama-2-70b-chatEpoch: Llama-2-70b-chat, Meta AIopendefault41.6%not datedconceptualreasoning.ai

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method