Skip to content
Weights Not Walls
Menu

MATH level 5

Re-run by Epoch AI

The hardest level (5) of the MATH dataset of competition mathematics problems, each with a single final answer.

Kind
Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
Score shown
Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
Results
108 in all: 43 for open-weight models, 65 for closed ones. 31 linked to a model page here.

Best open-weight result: deepseek-ai/DeepSeek-R1-0528, 96.6%, number 9 of 108 results.

MATH level 5 results
#ModelWeightsSettingScoreDateSource
1gpt-5-2025-08-07_highOpenAIclosedeffort high98.1%±0.329 Oct 2025Epoch AI log
2gpt-5-2025-08-07_mediumOpenAIclosedeffort medium97.9%±0.320 Aug 2025Epoch AI log
3gpt-5-mini-2025-08-07_highOpenAIclosedeffort high97.8%±0.330 Oct 2025Epoch AI log
4o4-mini-2025-04-16_highOpenAIclosedeffort high97.8%±0.316 Apr 2025Epoch AI log
5o3-2025-04-16_highOpenAIclosedeffort high97.8%±0.316 Apr 2025Epoch AI log
6claude-sonnet-4-5-20250929_32KAnthropiccloseddefault97.7%±0.421 Oct 2025Epoch AI log
7qwen3-max-2025-09-23Alibabacloseddefault97.1%±0.49 Oct 2025Epoch AI log
8gpt-5-mini-2025-08-07_mediumOpenAIclosedeffort medium96.8%±0.420 Aug 2025Epoch AI log
9deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeekopendefault96.6%±0.429 May 2025Epoch AI log
10o3-mini-2025-01-31_highOpenAIclosedeffort high96.5%±0.413 Feb 2025Epoch AI log
11claude-haiku-4-5-20251001_32KAnthropiccloseddefault96.4%±0.522 Oct 2025Epoch AI log
12gemini-2.5-pro-preview-05-06Google DeepMindcloseddefault95.9%±0.48 May 2025Epoch AI log
13gemini-2.5-pro-preview-03-25Google DeepMindcloseddefault95.6%±0.47 May 2025Epoch AI log
14gpt-5-nano-2025-08-07_mediumOpenAIclosedeffort medium95.2%±0.520 Aug 2025Epoch AI log
15o3-mini-2025-01-31_mediumOpenAIclosedeffort medium95.2%±0.531 Jan 2025Epoch AI log
16gpt-5-nano-2025-08-07_highOpenAIclosedeffort high94.9%±0.520 Aug 2025Epoch AI log
17o1-2024-12-17_highOpenAIclosedeffort high94.7%±0.613 Feb 2025Epoch AI log
18o1-2024-12-17_mediumOpenAIclosedeffort medium94.4%±0.627 Jan 2025Epoch AI log
19deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeekopendefault93.1%±0.731 Jan 2025Epoch AI log
20claude-3-7-sonnet-20250219_64KAnthropiccloseddefault91.2%±0.813 Mar 2025Epoch AI log
21grok-3-mini-beta_lowxAIclosedeffort low90.9%±0.810 Apr 2025Epoch AI log
22claude-3-7-sonnet-20250219_32KAnthropiccloseddefault90.0%±0.812 Mar 2025Epoch AI log
23deepseek-ai/DeepSeek-R1-Distill-Llama-70BEpoch: DeepSeek-R1-Distill-Llama-70B, DeepSeekopendefault89.9%±0.710 Mar 2025Epoch AI log
24o1-mini-2024-09-12_highOpenAIclosedeffort high89.2%±0.713 Feb 2025Epoch AI log
25grok-3-betaxAIcloseddefault88.7%±0.910 Apr 2025Epoch AI log
26grok-3-mini-beta_highxAIclosedeffort high88.1%±0.910 Apr 2025Epoch AI log
27gpt-4.1-mini-2025-04-14OpenAIcloseddefault87.3%±0.714 Apr 2025Epoch AI log
28deepseek-ai/DeepSeek-R1-Distill-Qwen-14BEpoch: DeepSeek-R1-Distill-Qwen-14B, DeepSeekopendefault87.1%±0.711 Mar 2025Epoch AI log
29claude-haiku-4-5-20251001Anthropiccloseddefault86.9%±0.816 Oct 2025Epoch AI log
30claude-3-7-sonnet-20250219_16KAnthropiccloseddefault86.3%±0.926 Feb 2025Epoch AI log
31claude-opus-4-20250514Anthropiccloseddefault85.0%±1.022 May 2025Epoch AI log
32claude-sonnet-4-20250514Anthropiccloseddefault84.4%±1.022 May 2025Epoch AI log
33o1-mini-2024-09-12_mediumOpenAIclosedeffort medium84.3%±0.827 Jan 2025Epoch AI log
34gemini-2.0-pro-exp-02-05Google DeepMindcloseddefault83.5%±1.06 Feb 2025Epoch AI log
35gpt-4.1-2025-04-14OpenAIcloseddefault83.0%±0.814 Apr 2025Epoch AI log
36gemini-2.0-flash-001Google DeepMind,Googlecloseddefault82.2%±0.96 Feb 2025Epoch AI log
37o1-preview-2024-09-12OpenAIcloseddefault81.6%±1.127 Jan 2025Epoch AI log
38mistral-medium-2505Mistral AIcloseddefault81.6%±0.97 May 2025Epoch AI log
39gpt-4.5-preview-2025-02-27OpenAIcloseddefault78.6%±1.128 Feb 2025Epoch AI log
40deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeekopendefault75.5%±0.91 Apr 2025Epoch AI log
41google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMindopendefault74.0%±1.113 Mar 2025Epoch AI log
42meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AIopendefault73.0%±1.18 Apr 2025Epoch AI log
43gemini-1.5-pro-002Google DeepMindcloseddefault70.4%±1.027 Jan 2025Epoch AI log
44gpt-4.1-nano-2025-04-14OpenAIcloseddefault70.0%±1.014 Apr 2025Epoch AI log
45Qwen/Qwen3-235B-A22BEpoch: qwen3-235b-a22b, Alibabaopendefault68.9%±0.93 Jun 2025Epoch AI log
46claude-3-7-sonnet-20250219Anthropiccloseddefault68.2%±1.124 Feb 2025Epoch AI log
47qwen-max-2025-01-25Alibabacloseddefault67.2%±1.11 Apr 2025Epoch AI log
48qwen-plus-2025-01-25Alibabacloseddefault65.3%±1.17 Apr 2025Epoch AI log
49microsoft/phi-4Epoch: phi-4, Microsoft Researchopendefault64.9%±1.131 Jan 2025Epoch AI log
50deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopendefault64.9%±1.027 Jan 2025Epoch AI log
51grok-2-1212xAIcloseddefault63.5%±1.317 Feb 2025Epoch AI log
52Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibabaopendefault63.2%±1.127 Jan 2025Epoch AI log
53meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AIopendefault62.3%±1.28 Apr 2025Epoch AI log
54gemini-1.5-flash-002Google DeepMindcloseddefault61.9%±1.127 Jan 2025Epoch AI log
55claude-3-5-sonnet-20241022Anthropiccloseddefault56.9%±1.127 Jan 2025Epoch AI log
56qwen-turbo-2024-11-01Alibabacloseddefault56.2%±1.17 Apr 2025Epoch AI log
57Qwen/Qwen2.5-32B-InstructEpoch: qwen2.5-32b-instruct, Alibabaopendefault56.1%±1.130 Jan 2025Epoch AI log
58gpt-4o-2024-08-06OpenAIcloseddefault53.3%±1.127 Jan 2025Epoch AI log
59gpt-4o-mini-2024-07-18OpenAIcloseddefault52.6%±1.127 Jan 2025Epoch AI log
60claude-3-5-sonnet-20240620Anthropiccloseddefault51.7%±1.227 Jan 2025Epoch AI log
61gpt-4o-2024-05-13OpenAIcloseddefault51.0%±1.127 Jan 2025Epoch AI log
62mistral-large-2411Mistral AIno page matchopendefault50.3%±1.125 Feb 2025Epoch AI log
63gpt-4o-2024-11-20OpenAIcloseddefault49.8%±1.15 Feb 2025Epoch AI log
64meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AIopendefault49.8%±1.227 Jan 2025Epoch AI log
65mistral-small-2503Mistral AIno page matchopendefault46.8%±1.218 Mar 2025Epoch AI log
66gpt-4-turbo-2024-04-09OpenAIcloseddefault46.7%±1.127 Feb 2025Epoch AI log
67claude-3-5-haiku-20241022Anthropiccloseddefault46.4%±1.212 Mar 2025Epoch AI log
68mistral-large-2407Mistral AIno page matchopendefault44.8%±1.127 Jan 2025Epoch AI log
69mistral-small-2501Mistral AIno page matchopendefault44.8%±1.130 Jan 2025Epoch AI log
70allenai/Llama-3.1-Tulu-3-70B-DPOEpoch: Llama-3.1-Tulu-3-70B-DPO, Allen Institute for AI,University of Washingtonopendefault42.7%±1.027 Jan 2025Epoch AI log
71meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopendefault41.6%±1.227 Jan 2025Epoch AI log
72gemini-1.5-pro-001Google DeepMindcloseddefault40.7%±1.127 Jan 2025Epoch AI log
73gpt-4-1106-previewOpenAIcloseddefault40.0%±1.127 Jan 2025Epoch AI log
74meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AIopendefault39.4%±1.027 Jan 2025Epoch AI log
75Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, Alibabaopendefault39.1%±1.027 Jan 2025Epoch AI log
76claude-3-opus-20240229Anthropiccloseddefault37.5%±1.127 Jan 2025Epoch AI log
77meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AIopendefault36.7%±1.027 Jan 2025Epoch AI log
78gpt-4-0125-previewOpenAIcloseddefault35.4%±1.127 Jan 2025Epoch AI log
79google/gemma-2-27b-itEpoch: gemma-2-27b-it, Google DeepMindopendefault27.9%±1.027 Jan 2025Epoch AI log
80WizardLM-2-8x22BMicrosoftno page matchopendefault25.7%±0.927 Jan 2025Epoch AI log
8101-ai/Yi-1.5-34B-ChatEpoch: Yi-1.5-34B-Chat, 01.AIopendefault25.5%±0.927 Jan 2025Epoch AI log
82gemini-1.5-flash-001Google DeepMindcloseddefault25.1%±0.927 Jan 2025Epoch AI log
83mistral-large-2402Mistral AIcloseddefault24.5%±0.927 Jan 2025Epoch AI log
84open-mixtral-8x22bMistral AIno page matchopendefault24.2%±0.827 Jan 2025Epoch AI log
85gpt-4-0613OpenAIcloseddefault23.0%±0.827 Jan 2025Epoch AI log
86meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopendefault22.9%±0.927 Jan 2025Epoch AI log
87Hermes-2-Theta-Llama-3-70BNous Research,Arcee AIno page matchopendefault22.7%±0.827 Jan 2025Epoch AI log
88meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AIopendefault22.6%±0.827 Jan 2025Epoch AI log
89google/gemma-2-9b-itEpoch: gemma-2-9b-it, Google DeepMindopendefault21.0%±0.927 Jan 2025Epoch AI log
90claude-3-sonnet-20240229Anthropiccloseddefault18.2%±0.827 Jan 2025Epoch AI log
91microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, Microsoftopendefault17.6%±0.731 Jan 2025Epoch AI log
92gpt-3.5-turbo-1106OpenAIcloseddefault15.9%±0.727 Jan 2025Epoch AI log
93ministral-8b-2410Mistral AIno page matchopendefault14.9%±0.727 Jan 2025Epoch AI log
94claude-3-haiku-20240307Anthropiccloseddefault14.9%±0.727 Jan 2025Epoch AI log
95ministral-3b-2410Mistral AIcloseddefault14.4%±0.727 Jan 2025Epoch AI log
96claude-2.0Anthropiccloseddefault11.7%±0.727 Jan 2025Epoch AI log
97dbrx-instructDatabricksno page matchopendefault11.7%±0.727 Jan 2025Epoch AI log
98gpt-3.5-turbo-0125OpenAIcloseddefault11.6%±0.627 Jan 2025Epoch AI log
99gemini-1.0-pro-001Google DeepMindcloseddefault11.2%±0.627 Jan 2025Epoch AI log
100open-mistral-nemo-2407Mistral AIno page matchopendefault10.8%±0.627 Jan 2025Epoch AI log
101open-mixtral-8x7bMistral AIno page matchopendefault10.0%±0.527 Jan 2025Epoch AI log
102mistralai/Mixtral-8x7B-Instruct-v0.1Epoch: Mixtral-8x7B-Instruct-v0.1, Mistral AIopendefault9.3%±0.527 Jan 2025Epoch AI log
103deepseek-ai/deepseek-llm-67b-chatEpoch: deepseek-llm-67b-chat, DeepSeekopendefault6.4%±0.427 Jan 2025Epoch AI log
104meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AIopendefault6.1%±0.427 Jan 2025Epoch AI log
10501-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AIopendefault5.1%±0.327 Jan 2025Epoch AI log
106open-mistral-7bMistral AIno page matchopendefault3.7%±0.327 Jan 2025Epoch AI log
107mistralai/Mistral-7B-Instruct-v0.3Epoch: Mistral-7B-Instruct-v0.3, Mistral AIopendefault3.6%±0.327 Jan 2025Epoch AI log
108meta-llama/Llama-2-70b-chat-hfEpoch: Llama-2-70b-chat-hf, Meta AIopendefault3.3%±0.327 Jan 2025Epoch AI log

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method