Skip to content
Weights Not Walls
Menu

OTIS Mock AIME 2024-2025

Re-run by Epoch AI

Mock exams in the style of the AIME, a US mathematics competition, where every answer is a whole number.

Kind
Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
Score shown
Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.1%. Error bars as the source gives them.
Results
318 in all: 113 for open-weight models, 201 for closed ones, 4 where Epoch does not say. 99 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K3.1-Pro, 100.0%, number 16 of 318 results.

OTIS Mock AIME 2024-2025 results
#ModelWeightsSettingScoreDateSource
1claude-fable-5_highAnthropicclosedeffort high100.0%6 Aug 2026Epoch AI log
2claude-fable-5-1_maxAnthropicclosedeffort max100.0%1 Sep 2026Epoch AI log
3claude-opus-5-5_maxAnthropicclosedeffort max100.0%22 Sep 2026Epoch AI log
4claude-sonnet-5-5_maxAnthropicclosedeffort max100.0%29 Sep 2026Epoch AI log
5gemini-4-argon_highGoogle DeepMindclosedeffort high100.0%13 Oct 2026Epoch AI log
6gemini-4-carbon_highGoogle DeepMindclosedeffort high100.0%2 Nov 2026Epoch AI
7gemini-4-hydrogen_xhighGoogle DeepMindclosedeffort xhigh100.0%13 Nov 2026Epoch AI log
8gpt-5.5-pre-release_xhighOpenAIclosedeffort xhigh100.0%24 Apr 2026Epoch AI log
9gpt-5.5-pro-pre-release_xhighOpenAIclosedeffort xhigh100.0%24 Apr 2026Epoch AI log
10gpt-5.6-sol_maxOpenAIclosedeffort max100.0%9 Jul 2026Epoch AI log
11gpt-6-astra_maxOpenAIclosedeffort max100.0%30 Aug 2026Epoch AI log
12gpt-6-sol_maxOpenAIclosedeffort max100.0%22 Sep 2026Epoch AI log
13gpt-6.1-sol_maxOpenAIclosedeffort max100.0%29 Sep 2026Epoch AI log
14gpt-6.1-terra_maxOpenAIclosedeffort max100.0%26 Oct 2026Epoch AI log
15gpt-6.2-astra_maxOpenAIclosedeffort max100.0%27 Oct 2026Epoch AI
16moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro-together_max, Moonshotopeneffort max100.0%6 Nov 2026Epoch AI log
17qwen3.8-max-0902_xhighAlibabaclosedeffort xhigh100.0%2 Sep 2026Epoch AI log
18claude-fable-5_maxAnthropicclosedeffort max99.7%±0.310 Jun 2026Epoch AI log
19claude-fable-5-2_maxAnthropicclosedeffort max99.7%±0.36 Nov 2026Epoch AI
20gpt-5.6-terra_maxOpenAIclosedeffort max99.7%±0.39 Jul 2026Epoch AI log
21deepseek-ai/DeepSeek-V4.1-Flash-1017Epoch: deepseek-v4.1-flash-1017_max, DeepSeekopeneffort max99.4%±0.419 Oct 2026Epoch AI log
22mistral-large-4-0-oss_highMistral AIno page matchopeneffort high99.4%±0.42 Nov 2026Epoch AI log
23muse-wonder-1.0_maxMeta AIclosedeffort max99.4%±0.412 Oct 2026Epoch AI log
24qwen3.8-max_xhighAlibabaclosedeffort xhigh99.4%±0.44 Aug 2026Epoch AI log
25doubao-seed-evolving_highByteDanceclosedeffort high99.2%±0.52 Nov 2026Epoch AI log
26grok-4.6_xhighxAIclosedeffort xhigh99.2%±0.514 Aug 2026Epoch AI log
27muse-spark-1.3_xhighMeta AIclosedeffort xhigh99.2%±0.617 Sep 2026Epoch AI log
28claude-haiku-5-5_xhighAnthropicclosedeffort xhigh98.9%±0.57 Oct 2026Epoch AI log
29claude-opus-5_maxAnthropicclosedeffort max98.9%±1.124 Jul 2026Epoch AI log
30gemini-3.8-flash_highGoogle DeepMindclosedeffort high98.9%±0.92 Sep 2026Epoch AI log
31gpt-6-luna_maxOpenAIclosedeffort max98.9%±0.722 Sep 2026Epoch AI log
32deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_max, DeepSeekopeneffort max98.6%±1.018 Aug 2026Epoch AI log
33claude-opus-4-8_maxAnthropicclosedeffort max98.3%±1.47 Jun 2026Epoch AI log
34deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_max, DeepSeekopeneffort max98.3%±0.68 Oct 2026Epoch AI log
35gpt-5.6-luna_maxOpenAIclosedeffort max98.3%±1.29 Jul 2026Epoch AI log
36mistral-large-4-0_highMistral AIno page matchopeneffort high98.3%±0.612 Oct 2026Epoch AI log
37grok-4.7_xhighxAIclosedeffort xhigh98.1%±1.422 Sep 2026Epoch AI log
38meta-models/Muse-Spark-Open-816B-A42BEpoch: Muse-Spark-Open-816B-A42B_xhigh, Meta AIopeneffort xhigh98.1%±1.315 Oct 2026Epoch AI log
39claude-opus-4-7_xhighAnthropicclosedeffort xhigh97.8%±2.217 Apr 2026Epoch AI log
40claude-fable-5_lowAnthropicclosedeffort low97.8%±2.26 Aug 2026Epoch AI log
41claude-opus-4-8_lowAnthropicclosedeffort low97.8%±2.26 Aug 2026Epoch AI log
42claude-opus-5Anthropiccloseddefault97.8%±2.26 Aug 2026Epoch AI log
43gpt-5.4-2026-03-05_highOpenAIclosedeffort high97.8%±2.215 Jul 2026Epoch AI log
44grok-4.5_highxAIclosedeffort high97.8%±1.38 Jul 2026Epoch AI log
45grok-4.6_highxAIclosedeffort high97.8%±1.012 Aug 2026Epoch AI log
46claude-haiku-5-5_highAnthropicclosedeffort high97.2%±1.37 Oct 2026Epoch AI log
47gemini-3.7-flash_highGoogle DeepMindclosedeffort high97.2%±1.914 Aug 2026Epoch AI log
48moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshotopeneffort max97.2%±1.116 Jul 2026Epoch AI log
49deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeekopeneffort max96.7%±2.017 Jun 2026Epoch AI log
50doubao-seed-2.1-turbo-260628_highByteDanceclosedeffort high96.1%±2.530 Oct 2026Epoch AI log
51gpt-5.2-2025-12-11_highOpenAIclosedeffort high96.1%±2.611 Dec 2025Epoch AI log
52moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault96.1%±2.42 May 2026Epoch AI log
53gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh96.1%±2.713 Dec 2025Epoch AI log
54gemini-3.1-pro-previewGoogle DeepMindcloseddefault95.6%±3.120 Feb 2026Epoch AI log
55deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_high, DeepSeekopeneffort high95.6%±3.16 Aug 2026Epoch AI log
56doubao-seed-2.1-pro-260628_highByteDanceclosedeffort high95.6%±2.930 Oct 2026Epoch AI log
57gemini-3-flash-preview_highGoogle DeepMindclosedeffort high95.6%±3.16 Aug 2026Epoch AI log
58gemini-3.1-pro-preview_highGoogle DeepMindclosedeffort high95.6%±3.16 Aug 2026Epoch AI log
59gemini-3.5-flash_highGoogle DeepMindclosedeffort high95.6%±2.725 May 2026Epoch AI log
60gpt-5.4-2026-03-05_mediumOpenAIclosedeffort medium95.6%±3.115 Jul 2026Epoch AI log
61gpt-5.6-sol_lowOpenAIclosedeffort low95.6%±3.17 Aug 2026Epoch AI log
62moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, Moonshotopendefault95.6%±3.17 Aug 2026Epoch AI log
63qwen3.7-maxAlibabacloseddefault95.6%±3.17 Aug 2026Epoch AI log
64gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh95.3%±3.26 Mar 2026Epoch AI log
65claude-sonnet-5_xhighAnthropicclosedeffort xhigh94.7%±2.31 Jul 2026Epoch AI log
66claude-opus-4-6_64KAnthropiccloseddefault94.4%±2.86 Feb 2026Epoch AI log
67deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeekopeneffort max94.4%±2.12 Aug 2026Epoch AI log
68gemini-3.6-flash_highGoogle DeepMindclosedeffort high94.2%±3.12 Aug 2026Epoch AI log
69doubao-seed-2.0-pro-260215_highByteDanceclosedeffort high93.9%±2.730 Oct 2026Epoch AI log
70zai-org/GLM-5.3-FlashEpoch: glm-5.3-flash_max, Z.ai (Zhipu AI)openeffort max93.9%±2.226 Aug 2026Epoch AI log
71gpt-5.2-2025-12-11_mediumOpenAIclosedeffort medium93.9%±3.111 Dec 2025Epoch AI log
72claude-opus-5_lowAnthropicclosedeffort low93.3%±3.86 Aug 2026Epoch AI log
73zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI)opendefault93.3%±3.810 Aug 2026Epoch AI log
74grok-4.3_highxAIclosedeffort high93.3%±3.017 Jun 2026Epoch AI log
75moonshotai/Kimi-K3Epoch: kimi-k3_high, Moonshotopeneffort high93.3%±3.87 Aug 2026Epoch AI log
76qwen3.6-plusAlibabacloseddefault93.3%±3.87 Aug 2026Epoch AI log
77qwen3.7-plusAlibabacloseddefault93.3%±3.87 Aug 2026Epoch AI log
78claude-opus-4-6_32KAnthropiccloseddefault93.1%±3.36 Feb 2026Epoch AI log
79gemini-3-flash-previewGoogle DeepMindcloseddefault92.8%±3.517 Dec 2025Epoch AI log
80grok-4.20-0309-reasoningxAIcloseddefault92.2%±3.313 Jul 2026Epoch AI log
81moonshotai/Kimi-K2.5Epoch: fireworks/kimi-k2p5, Moonshotopendefault92.2%±3.92 Feb 2026Epoch AI log
82gemini-3-pro-previewGoogle DeepMindcloseddefault91.4%±3.719 Nov 2025Epoch AI log
83gpt-5-2025-08-07_highOpenAIclosedeffort high91.4%±3.729 Oct 2025Epoch AI log
84claude-opus-4-6_maxAnthropicclosedeffort max91.1%±4.36 Aug 2026Epoch AI log
85doubao-seed-2.0-lite-260428_highByteDanceclosedeffort high91.1%±3.630 Oct 2026Epoch AI log
86zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI)openeffort max91.1%±3.524 Aug 2026Epoch AI log
87Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibabaopendefault91.1%±4.37 Aug 2026Epoch AI log
88qwen3.6-max-previewAlibabanot stateddefault91.1%±4.37 Aug 2026Epoch AI log
89thinkingmachines/Inkling-SmallEpoch: inkling-small_xhigh, Thinking Machinesopeneffort xhigh90.0%±4.014 Aug 2026Epoch AI log
90muse-sparkMeta AIcloseddefault88.9%±4.78 Apr 2026Epoch AI
91gemini-3.5-flash_lowGoogle DeepMindclosedeffort low88.9%±4.76 Aug 2026Epoch AI log
92gpt-5.4-mini-2026-03-17_xhighOpenAIclosedeffort xhigh88.9%±4.77 Aug 2026Epoch AI log
93gpt-5.6-terra_lowOpenAIclosedeffort low88.9%±4.77 Aug 2026Epoch AI log
94thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machinesopeneffort xhigh88.9%±3.95 Aug 2026Epoch AI log
95openai/gpt-oss-120bEpoch: openai/gpt-oss-120b_high, OpenAIopeneffort high88.9%±4.511 Dec 2025Epoch AI log
96Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibabaopendefault88.9%±4.77 Aug 2026Epoch AI log
97gpt-5.1-2025-11-13_highOpenAIclosedeffort high88.6%±4.013 Nov 2025Epoch AI log
98deepseek-reasonerDeepSeekno page matchopendefault87.8%±4.116 Dec 2025Epoch AI log
99gpt-5.4-nano-2026-03-17_highOpenAIclosedeffort high87.8%±3.914 Apr 2026Epoch AI log
100gpt-5-2025-08-07_mediumOpenAIclosedeffort medium87.2%±3.97 Aug 2025Epoch AI log
101gpt-5.4-mini-2026-03-17_highOpenAIclosedeffort high87.2%±4.315 Apr 2026Epoch AI log
102claude-opus-4-7_maxAnthropicclosedeffort max86.7%±5.16 Aug 2026Epoch AI log
103gpt-5-mini-2025-08-07_highOpenAIclosedeffort high86.7%±4.330 Oct 2025Epoch AI log
104nemotron-3-ultraNvidiano page matchopendefault86.7%±5.110 Aug 2026Epoch AI log
105Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault86.7%±5.110 Dec 2025Epoch AI log
106qwen3.5-plusAlibabanot stateddefault86.7%±5.17 Aug 2026Epoch AI log
107Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibabaopendefault86.7%±5.17 Aug 2026Epoch AI log
108qwen3.7-flashAlibabacloseddefault86.7%±5.17 Aug 2026Epoch AI log
109zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI)openeffort max86.4%±4.225 Jun 2026Epoch AI log
110claude-opus-4-5-20251101_32KAnthropiccloseddefault86.1%±4.324 Nov 2025Epoch AI log
111claude-sonnet-4-6_32KAnthropiccloseddefault85.8%±4.520 Feb 2026Epoch AI log
112gpt-5.1-2025-11-13_mediumOpenAIclosedeffort medium85.6%±4.417 Nov 2025Epoch AI log
113gemini-2.5-proGoogle DeepMindcloseddefault84.7%±4.816 Nov 2025Epoch AI log
114claude-opus-4-8_noneAnthropicclosedeffort none84.4%±5.56 Aug 2026Epoch AI log
115gpt-5.4-2026-03-05_lowOpenAIclosedeffort low84.4%±5.515 Jul 2026Epoch AI log
116gpt-5.5_lowOpenAIclosedeffort low84.4%±5.57 Aug 2026Epoch AI log
117o3-2025-04-16_mediumOpenAIclosedeffort medium84.4%±5.57 Aug 2026Epoch AI log
118qwen3.5-flashAlibabanot stateddefault84.4%±5.57 Aug 2026Epoch AI log
119qwen3.6-flashAlibabanot stateddefault84.4%±5.57 Aug 2026Epoch AI log
120grok-4-0709xAIcloseddefault84.0%±5.0not datedEpoch AI
121o3-2025-04-16_highOpenAIclosedeffort high83.9%±4.416 Apr 2025Epoch AI log
122zai-org/GLM-4.7Epoch: glm-4.7, Z.ai (Zhipu AI)opendefault83.3%±4.829 Jan 2026Epoch AI log
123kimi-k2-thinking-turboMoonshotno page matchopendefault83.1%±4.813 Nov 2025Epoch AI log
124claude-sonnet-4-6_mediumAnthropicclosedeffort medium82.2%±5.813 Jul 2026Epoch AI log
125gemini-3.6-flash_lowGoogle DeepMindclosedeffort low82.2%±5.87 Aug 2026Epoch AI log
126google/gemma-4-26B-A4B-itEpoch: gemma-4-26b-a4b-it_minimal, Google DeepMindopeneffort minimal82.2%±5.828 Aug 2026Epoch AI log
127Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b_none, Alibabaopeneffort none82.2%±5.87 Aug 2026Epoch AI log
128claude-opus-4-5-20251101_16KAnthropiccloseddefault81.7%±5.024 Nov 2025Epoch AI log
129o4-mini-2025-04-16_highOpenAIclosedeffort high81.7%±4.716 Apr 2025Epoch AI log
130gpt-5-nano-2025-08-07_highOpenAIclosedeffort high81.1%±4.631 Oct 2025Epoch AI log
131claude-sonnet-5_maxAnthropicclosedeffort max80.0%±6.06 Aug 2026Epoch AI log
132gemini-3.1-flash-lite_highGoogleclosedeffort high80.0%±6.06 Aug 2026Epoch AI log
133gemini-3.5-flash_minimalGoogle DeepMindclosedeffort minimal80.0%±6.015 Jul 2026Epoch AI log
134gemini-3.6-flash_minimalGoogle DeepMindclosedeffort minimal80.0%±6.07 Aug 2026Epoch AI log
135zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI)opendefault80.0%±6.012 Feb 2026Epoch AI log
136qwen3.7-plus_noneAlibabaclosedeffort none80.0%±6.07 Aug 2026Epoch AI log
137gpt-5.2-2025-12-11_lowOpenAIclosedeffort low78.9%±5.011 Dec 2025Epoch AI log
138gpt-5-mini-2025-08-07_mediumOpenAIclosedeffort medium78.3%±4.87 Aug 2025Epoch AI log
139claude-sonnet-4-5-20250929_32KAnthropiccloseddefault77.8%±6.321 Oct 2025Epoch AI log
140claude-sonnet-4-5-20250929_59KAnthropiccloseddefault77.8%±6.328 Oct 2025Epoch AI log
141grok-3-mini-beta_highxAIclosedeffort high77.8%±6.310 Apr 2025Epoch AI log
142qwen3.7-flash_noneAlibabaclosedeffort none77.8%±6.37 Aug 2026Epoch AI log
143o3-mini-2025-01-31_highOpenAIclosedeffort high76.9%±5.227 Feb 2025Epoch AI log
144claude-sonnet-4-6_highAnthropicclosedeffort high75.6%±6.513 Jul 2026Epoch AI log
145zai-org/GLM-5.2Epoch: glm-5.2_low, Z.ai (Zhipu AI)openeffort low75.6%±6.510 Aug 2026Epoch AI log
146gpt-5-nano-2025-08-07_mediumOpenAIclosedeffort medium74.2%±5.27 Aug 2025Epoch AI log
147google/gemma-4-31B-itEpoch: gemma-4-31b-it_minimal, Google DeepMindopeneffort minimal73.3%±6.76 Aug 2026Epoch AI log
148o1-2024-12-17_highOpenAIclosedeffort high73.3%±6.715 Jul 2026Epoch AI log
149o1-2024-12-17_mediumOpenAIclosedeffort medium73.3%±6.727 Feb 2025Epoch AI log
150o4-mini-2025-04-16_mediumOpenAIclosedeffort medium73.3%±6.77 Aug 2026Epoch AI log
151qwen3-max-2025-09-23Alibabacloseddefault73.3%±5.96 Oct 2025Epoch AI log
152gemini-2.5-flash-preview-04-17Google DeepMindcloseddefault73.1%±5.522 Apr 2025Epoch AI log
153claude-sonnet-4-20250514_32KAnthropiccloseddefault71.1%±6.822 May 2025Epoch AI log
154claude-sonnet-4-5-20250929_16KAnthropiccloseddefault71.1%±6.828 Oct 2025Epoch AI log
155claude-sonnet-4-6_maxAnthropicclosedeffort max71.1%±6.86 Aug 2026Epoch AI log
156gemini-3.5-flash-lite_highGoogle DeepMindclosedeffort high71.1%±6.86 Aug 2026Epoch AI log
157MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMaxopendefault71.1%±6.810 Aug 2026Epoch AI log
158gemini-2.5-flash-preview-05-20Google DeepMindcloseddefault70.8%±5.720 May 2025Epoch AI log
159Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibabaopendefault70.3%±6.030 Aug 2026Epoch AI log
160Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b_none, Alibabaopeneffort none70.0%±6.030 Aug 2026Epoch AI log
161seed-1.6ByteDancecloseddefault69.4%±6.130 Oct 2026Epoch AI log
162claude-opus-4-1-20250805_27KAnthropiccloseddefault68.9%±7.05 Aug 2025Epoch AI log
163claude-sonnet-4-20250514_59KAnthropiccloseddefault68.9%±7.023 May 2025Epoch AI log
164gpt-5.4-nano-2026-03-17_lowOpenAIclosedeffort low68.9%±7.07 Aug 2026Epoch AI log
165gpt-5.6-sol_noneOpenAIclosedeffort none68.9%±7.07 Aug 2026Epoch AI log
166moonshotai/Kimi-K3Epoch: kimi-k3_low, Moonshotopeneffort low68.9%±7.07 Aug 2026Epoch AI log
167Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b_none, Alibabaopeneffort none68.9%±7.07 Aug 2026Epoch AI log
168gpt-5.5-instantOpenAIcloseddefault68.1%±6.22 Aug 2026Epoch AI log
169ByteDance-Seed/Seed-OSS-36B-InstructEpoch: seed-oss-36b-instructopendefault67.5%±6.328 Aug 2026Epoch AI log
170Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibabaopendefault66.9%±6.030 Aug 2026Epoch AI log
171claude-haiku-4-5-20251001_32KAnthropiccloseddefault66.7%±7.122 Oct 2025Epoch AI log
172gpt-5.6-luna_lowOpenAIclosedeffort low66.7%±7.17 Aug 2026Epoch AI log
173Qwen/Qwen3.6-27BEpoch: qwen3.6-27b_none, Alibabaopeneffort none66.7%±7.17 Aug 2026Epoch AI log
174deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeekopendefault66.4%±4.929 May 2025Epoch AI log
175Qwen/Qwen3-14BEpoch: qwen3-14b, Alibabaopendefault66.4%±6.228 Aug 2026Epoch AI log
176openai/gpt-oss-20bEpoch: gpt-oss-20b_medium, OpenAIopeneffort medium65.3%±5.927 Aug 2026Epoch AI log
177claude-opus-4-1-20250805_16KAnthropiccloseddefault64.4%±7.25 Aug 2025Epoch AI log
178claude-opus-4-20250514_27KAnthropiccloseddefault64.4%±7.228 May 2025Epoch AI log
179gpt-5.1-2025-11-13_lowOpenAIclosedeffort low63.9%±5.925 Nov 2025Epoch AI log
180o3-mini-2025-01-31_mediumOpenAIclosedeffort medium63.9%±6.025 Feb 2025Epoch AI log
181Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibabaopendefault62.8%±6.228 Aug 2026Epoch AI log
182gpt-5.2-2025-12-11_noneOpenAIclosedeffort none62.2%±7.313 Jul 2026Epoch AI log
183grok-3-mini-beta_lowxAIclosedeffort low62.2%±7.310 Apr 2025Epoch AI log
184Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibabaopendefault62.2%±6.530 Aug 2026Epoch AI log
185Qwen/Qwen3.5-9BEpoch: qwen3.5-9B_none, Alibabaopeneffort none61.7%±6.027 Aug 2026Epoch AI log
186claude-opus-4-20250514_16KAnthropiccloseddefault60.0%±7.422 May 2025Epoch AI log
187gemini-3.5-flash-lite_lowGoogle DeepMindclosedeffort low60.0%±7.46 Aug 2026Epoch AI log
188o3-2025-04-16_lowOpenAIclosedeffort low60.0%±7.415 Jul 2026Epoch AI log
189Qwen/QwQ-32BEpoch: QwQ-32B, Alibabaopendefault59.2%±6.328 Aug 2026Epoch AI log
190zai-org/GLM-4.7-FlashEpoch: glm-4.7-flashopendefault58.3%±6.630 Aug 2026Epoch AI log
191claude-3-7-sonnet-20250219_64KAnthropiccloseddefault57.8%±7.413 Mar 2025Epoch AI log
192gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Googlecloseddefault57.8%±7.47 Mar 2025Epoch AI log
193gpt-5.4-2026-03-05_noneOpenAIclosedeffort none57.8%±7.415 Jul 2026Epoch AI log
194gpt-5.5_noneOpenAIclosedeffort none57.8%±7.47 Aug 2026Epoch AI log
195o4-mini-2025-04-16_lowOpenAIclosedeffort low57.8%±7.413 Jul 2026Epoch AI log
196Qwen/Qwen3-8BEpoch: qwen3-8b, Alibabaopendefault56.1%±6.527 Aug 2026Epoch AI log
197Qwen/Qwen3.5-4BEpoch: qwen3.5-4b_none, Alibabaopeneffort none55.8%±6.228 Aug 2026Epoch AI log
198deepseek-ai/DeepSeek-R1-Distill-Qwen-32BEpoch: DeepSeek-R1-Distill-Qwen-32B, DeepSeekopendefault55.6%±6.228 Aug 2026Epoch AI log
199gpt-5-mini-2025-08-07_minimalOpenAIclosedeffort minimal55.6%±7.57 Aug 2026Epoch AI log
200grok-3-betaxAIcloseddefault55.6%±7.510 Apr 2025Epoch AI log
201Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibabaopendefault54.4%±6.230 Aug 2026Epoch AI log
202claude-3-7-sonnet-20250219_32KAnthropiccloseddefault53.3%±7.512 Mar 2025Epoch AI log
203claude-sonnet-4-20250514_16KAnthropiccloseddefault53.3%±7.522 May 2025Epoch AI log
204deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeekopendefault53.3%±7.526 Feb 2025Epoch AI log
205gpt-5.6-terra_noneOpenAIclosedeffort none53.3%±7.57 Aug 2026Epoch AI log
206o1-2024-12-17_lowOpenAIclosedeffort low53.3%±7.515 Jul 2026Epoch AI log
207Qwen/Qwen3-4B-Instruct-2507Epoch: qwen3-4b-instruct-2507opendefault52.2%±6.628 Aug 2026Epoch AI log
208deepseek-ai/DeepSeek-R1-Distill-Llama-70BEpoch: DeepSeek-R1-Distill-Llama-70B, DeepSeekopendefault51.4%±6.17 Mar 2025Epoch AI log
209gemini-3.5-flash-lite_minimalGoogle DeepMindclosedeffort minimal51.1%±7.56 Aug 2026Epoch AI log
210openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAIopeneffort high50.8%±6.727 Aug 2026Epoch AI log
211deepseek-ai/DeepSeek-R1-Distill-Qwen-14BEpoch: DeepSeek-R1-Distill-Qwen-14B, DeepSeekopendefault50.6%±6.028 Aug 2026Epoch AI log
212deepseek-chatDeepSeekno page matchopendefault48.9%±7.516 Jul 2026Epoch AI log
213claude-opus-4-5-20251101Anthropiccloseddefault48.1%±6.424 Nov 2025Epoch AI log
214o1-mini-2024-09-12_highOpenAIclosedeffort high46.9%±6.56 Mar 2025Epoch AI log
215claude-3-7-sonnet-20250219_16KAnthropiccloseddefault46.7%±7.526 Feb 2025Epoch AI log
216deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_none, DeepSeekopeneffort none46.7%±7.56 Aug 2026Epoch AI log
217gpt-5-2025-08-07_minimalOpenAIclosedeffort minimal46.7%±7.520 Jul 2026Epoch AI log
218gpt-5-nano-2025-08-07_lowOpenAIclosedeffort low46.7%±7.513 Jul 2026Epoch AI log
219gpt-5.4-nano-2026-03-17_noneOpenAIclosedeffort none46.7%±7.57 Aug 2026Epoch AI log
220gpt-4.1-mini-2025-04-14OpenAIcloseddefault44.7%±6.114 Apr 2025Epoch AI log
221o1-mini-2024-09-12_mediumOpenAIclosedeffort medium44.7%±6.36 Mar 2025Epoch AI log
222gemini-3.1-flash-lite_lowGoogleclosedeffort low44.4%±7.56 Aug 2026Epoch AI log
223o3-mini-2025-01-31_lowOpenAIclosedeffort low44.4%±7.520 Jul 2026Epoch AI log
224deepseek-ai/DeepSeek-R1-0528-Qwen3-8BEpoch: deepseek-r1-0528-qwen3-8bopendefault43.9%±6.828 Aug 2026Epoch AI log
225Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibabaopendefault42.8%±6.527 Aug 2026Epoch AI log
226claude-opus-4-20250514Anthropiccloseddefault42.2%±7.422 May 2025Epoch AI log
227openai/gpt-oss-20bEpoch: gpt-oss-20b_low, OpenAIopeneffort low40.3%±6.327 Aug 2026Epoch AI log
228claude-opus-4-1-20250805Anthropiccloseddefault40.0%±7.45 Aug 2025Epoch AI log
229gpt-5.6-luna_noneOpenAIclosedeffort none40.0%±7.47 Aug 2026Epoch AI log
230gpt-4.1-2025-04-14OpenAIcloseddefault38.3%±6.314 Apr 2025Epoch AI log
231deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeekopendefault37.8%±6.21 Apr 2025Epoch AI log
232gemini-3.1-flash-lite_minimalGoogleclosedeffort minimal37.8%±7.36 Aug 2026Epoch AI log
233gpt-4.5-preview-2025-02-27OpenAIcloseddefault37.8%±7.328 Feb 2025Epoch AI log
234gpt-5.1-2025-11-13_noneOpenAIclosedeffort none37.8%±7.37 Aug 2026Epoch AI log
235claude-haiku-4-5-20251001Anthropiccloseddefault35.8%±6.216 Oct 2025Epoch AI log
236claude-sonnet-4-5-20250929Anthropiccloseddefault35.6%±7.229 Sep 2025Epoch AI log
237gpt-5-nano-2025-08-07_minimalOpenAIclosedeffort minimal35.6%±7.213 Jul 2026Epoch AI log
238mistral-medium-2505Mistral AIcloseddefault32.2%±6.07 May 2025Epoch AI log
239gemini-2.0-flash-001Google DeepMind,Googlecloseddefault31.1%±6.325 Feb 2025Epoch AI log
240o1-preview-2024-09-12OpenAIcloseddefault31.1%±7.07 Mar 2025Epoch AI log
241mistral-small-3.2-2506Mistral AIno page matchopendefault30.3%±5.830 Aug 2026Epoch AI log
242mistralai/Magistral-Small-2506Epoch: magistral-small-2506, Mistral AIopendefault30.0%±5.96 Aug 2026Epoch AI log
243claude-sonnet-4-20250514Anthropiccloseddefault28.9%±6.822 May 2025Epoch AI log
244zai-org/GLM-5.2Epoch: glm-5.2_none, Z.ai (Zhipu AI)openeffort none28.9%±6.810 Aug 2026Epoch AI log
245gpt-4.1-nano-2025-04-14OpenAIcloseddefault28.9%±6.014 Apr 2025Epoch AI log
246mistralai/Magistral-Small-2509Epoch: magistral-small-2509, Mistral AIopendefault28.1%±5.630 Aug 2026Epoch AI log
247gpt-5.4-mini-2026-03-17_noneOpenAIclosedeffort none26.7%±6.77 Aug 2026Epoch AI log
248MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3_none, MiniMaxopeneffort none26.7%±6.710 Aug 2026Epoch AI log
249Qwen/Qwen3-14BEpoch: qwen3-14b_none, Alibabaopeneffort none25.8%±4.830 Aug 2026Epoch AI log
250Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B_none, Alibabaopeneffort none25.6%±5.230 Aug 2026Epoch AI log
251zai-org/GLM-4.7-FlashEpoch: glm-4.7-flash_noneopeneffort none25.0%±4.928 Aug 2026Epoch AI log
252gemini-1.5-pro-002Google DeepMindcloseddefault23.1%±5.225 Feb 2025Epoch AI log
253Qwen/Qwen3-32BEpoch: Qwen3-32B_none, Alibabaopeneffort none23.1%±5.130 Aug 2026Epoch AI log
254google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMindopendefault22.5%±5.327 Aug 2026Epoch AI log
255Qwen/Qwen3-8BEpoch: qwen3-8b_none, Alibabaopeneffort none22.2%±4.527 Aug 2026Epoch AI log
256claude-3-7-sonnet-20250219Anthropiccloseddefault21.9%±4.825 Feb 2025Epoch AI log
257deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5BEpoch: DeepSeek-R1-Distill-Qwen-1.5B, DeepSeekopendefault21.4%±5.328 Aug 2026Epoch AI log
258meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AIopendefault20.6%±4.98 Apr 2025Epoch AI log
259qwen-plus-2025-01-25Alibabacloseddefault17.8%±4.47 Apr 2025Epoch AI log
260google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMindopendefault16.7%±4.228 Aug 2026Epoch AI log
261gemini-1.5-flash-002Google DeepMindcloseddefault16.3%±4.125 Feb 2025Epoch AI log
262qwen-max-2025-01-25Alibabacloseddefault16.1%±4.11 Apr 2025Epoch AI log
263deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopendefault15.8%±4.325 Feb 2025Epoch AI log
264microsoft/phi-4Epoch: phi-4, Microsoft Researchopendefault13.8%±3.725 Feb 2025Epoch AI log
265grok-2-1212xAIcloseddefault11.5%±3.425 Feb 2025Epoch AI log
266meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AIopendefault9.7%±3.225 Feb 2025Epoch AI log
267claude-3-5-sonnet-20241022Anthropiccloseddefault8.5%±2.625 Feb 2025Epoch AI log
268mistral-large-2407Mistral AIno page matchopendefault8.5%±2.525 Feb 2025Epoch AI log
269Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibabaopendefault8.1%±2.625 Feb 2025Epoch AI log
270Qwen/Qwen3-1.7BEpoch: qwen3-1.7b_none, Alibabaopeneffort none8.1%±2.828 Aug 2026Epoch AI log
271meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AIopendefault7.8%±3.08 Apr 2025Epoch AI log
272mistral-large-2411Mistral AIno page matchopendefault7.8%±2.525 Feb 2025Epoch AI log
273google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMindopendefault7.5%±2.828 Aug 2026Epoch AI log
274Qwen/Qwen2.5-32B-InstructEpoch: qwen2.5-32b-instruct, Alibabaopendefault7.4%±2.125 Feb 2025Epoch AI log
275gpt-4o-mini-2024-07-18OpenAIcloseddefault6.9%±2.730 Jul 2025Epoch AI log
276gemini-1.5-pro-001Google DeepMindcloseddefault6.8%±1.925 Feb 2025Epoch AI log
277gpt-4-turbo-2024-04-09OpenAIcloseddefault6.7%±2.427 Feb 2025Epoch AI log
278mistral-small-3-2501Mistral AIno page matchopendefault6.7%±2.730 Aug 2026Epoch AI log
279claude-3-5-sonnet-20240620Anthropiccloseddefault6.5%±2.225 Feb 2025Epoch AI log
280gpt-4o-2024-08-06OpenAIcloseddefault6.4%±2.625 Feb 2025Epoch AI log
281gpt-4o-2024-05-13OpenAIcloseddefault6.3%±2.325 Feb 2025Epoch AI log
282gpt-4o-2024-11-20OpenAIcloseddefault6.3%±2.425 Feb 2025Epoch AI log
283qwen-turbo-2024-11-01Alibabacloseddefault6.1%±2.27 Apr 2025Epoch AI log
284mistral-small-2503Mistral AIno page matchopendefault5.8%±2.318 Mar 2025Epoch AI log
285mistral-small-2501Mistral AIno page matchopendefault5.3%±2.618 Mar 2025Epoch AI log
286meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopendefault5.1%±2.425 Feb 2025Epoch AI log
287claude-3-opus-20240229Anthropiccloseddefault4.7%±2.025 Feb 2025Epoch AI log
288gemini-1.5-flash-8b-001Google DeepMindcloseddefault4.6%±2.025 Feb 2025Epoch AI log
289allenai/Llama-3.1-Tulu-3-70B-DPOEpoch: Llama-3.1-Tulu-3-70B-DPO, Allen Institute for AI,University of Washingtonopendefault4.4%±1.87 Mar 2025Epoch AI log
290claude-3-5-haiku-20241022Anthropiccloseddefault4.3%±1.625 Feb 2025Epoch AI log
291meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AIopendefault4.3%±2.225 Feb 2025Epoch AI log
292gemini-1.5-flash-001Google DeepMindcloseddefault3.9%±1.425 Feb 2025Epoch AI log
293mistral-small-3.1-2503Mistral AIno page matchopendefault3.9%±1.530 Aug 2026Epoch AI log
294meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AIopendefault3.6%±1.725 Feb 2025Epoch AI log
295ibm-granite/granite-4.0-microEpoch: granite-4.0-microopendefault2.8%±1.430 Aug 2026Epoch AI log
296meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AIopendefault2.6%±0.925 Feb 2025Epoch AI log
297claude-2.0Anthropiccloseddefault2.5%±1.57 Mar 2025Epoch AI log
298claude-3-sonnet-20240229Anthropiccloseddefault2.5%±2.025 Feb 2025Epoch AI log
299Hermes-2-Theta-Llama-3-70BNous Research,Arcee AIno page matchopendefault2.5%±1.21 Apr 2025Epoch AI log
300Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibabaopendefault2.5%±0.930 Aug 2026Epoch AI log
301gpt-3.5-turbo-0125OpenAIcloseddefault2.2%±2.27 Aug 2026Epoch AI log
302claude-2.1Anthropiccloseddefault1.9%±1.07 Mar 2025Epoch AI log
303meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AIopendefault1.9%±1.030 Aug 2026Epoch AI log
304mistral-large-2402Mistral AIcloseddefault1.9%±0.925 Feb 2025Epoch AI log
305claude-3-haiku-20240307Anthropiccloseddefault1.8%±0.825 Feb 2025Epoch AI log
306meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopendefault1.7%±0.927 Aug 2026Epoch AI log
307google/gemma-2-27b-itEpoch: gemma-2-27b-it, Google DeepMindopendefault1.4%±0.97 Mar 2025Epoch AI log
308gemini-1.0-pro-001Google DeepMindcloseddefault1.1%±0.625 Feb 2025Epoch AI log
309google/gemma-3-1b-itEpoch: gemma-3-1b-it, Google DeepMindopendefault1.1%±0.528 Aug 2026Epoch AI log
310gpt-4-0613OpenAIcloseddefault1.1%±0.523 Oct 2025Epoch AI log
311deepseek-ai/deepseek-llm-67b-chatEpoch: deepseek-llm-67b-chat, DeepSeekopendefault0.8%±0.528 Aug 2026Epoch AI log
312ibm-granite/granite-4.0-1bEpoch: granite-4.0-1bopendefault0.8%±0.530 Aug 2026Epoch AI log
313google/gemma-2-9b-itEpoch: gemma-2-9b-it, Google DeepMindopendefault0.6%±0.47 Mar 2025Epoch AI log
314gpt-4-0314OpenAIcloseddefault0.6%±0.422 Oct 2025Epoch AI log
315meta-llama/Llama-3.2-1B-InstructEpoch: Llama-3.2-1B-Instruct, Meta AIopendefault0.6%±0.430 Aug 2026Epoch AI log
316ibm-granite/granite-4.0-350mEpoch: granite-4.0-350mopendefault0.3%±0.328 Aug 2026Epoch AI log
317mistralai/Mistral-7B-Instruct-v0.3Epoch: Mistral-7B-Instruct-v0.3, Mistral AIopendefault0.3%±0.330 Aug 2026Epoch AI log
318meta-llama/Llama-2-70b-chat-hfEpoch: Llama-2-70b-chat-hf, Meta AIopendefault0.0%25 Feb 2025Epoch AI log

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method