MATH level 5
Re-run by Epoch AIThe hardest level (5) of the MATH dataset of competition mathematics problems, each with a single final answer.
- Kind
- Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
- Score shown
- Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
- Results
- 108 in all: 43 for open-weight models, 65 for closed ones. 31 linked to a model page here.
Best open-weight result: deepseek-ai/DeepSeek-R1-0528, 96.6%, number 9 of 108 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 98.1%±0.3 | 29 Oct 2025 | Epoch AI log |
| 2 | gpt-5-2025-08-07_mediumOpenAI | closed | effort medium | 97.9%±0.3 | 20 Aug 2025 | Epoch AI log |
| 3 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 97.8%±0.3 | 30 Oct 2025 | Epoch AI log |
| 4 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 97.8%±0.3 | 16 Apr 2025 | Epoch AI log |
| 5 | o3-2025-04-16_highOpenAI | closed | effort high | 97.8%±0.3 | 16 Apr 2025 | Epoch AI log |
| 6 | claude-sonnet-4-5-20250929_32KAnthropic | closed | default | 97.7%±0.4 | 21 Oct 2025 | Epoch AI log |
| 7 | qwen3-max-2025-09-23Alibaba | closed | default | 97.1%±0.4 | 9 Oct 2025 | Epoch AI log |
| 8 | gpt-5-mini-2025-08-07_mediumOpenAI | closed | effort medium | 96.8%±0.4 | 20 Aug 2025 | Epoch AI log |
| 9 | deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeek | open | default | 96.6%±0.4 | 29 May 2025 | Epoch AI log |
| 10 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 96.5%±0.4 | 13 Feb 2025 | Epoch AI log |
| 11 | claude-haiku-4-5-20251001_32KAnthropic | closed | default | 96.4%±0.5 | 22 Oct 2025 | Epoch AI log |
| 12 | gemini-2.5-pro-preview-05-06Google DeepMind | closed | default | 95.9%±0.4 | 8 May 2025 | Epoch AI log |
| 13 | gemini-2.5-pro-preview-03-25Google DeepMind | closed | default | 95.6%±0.4 | 7 May 2025 | Epoch AI log |
| 14 | gpt-5-nano-2025-08-07_mediumOpenAI | closed | effort medium | 95.2%±0.5 | 20 Aug 2025 | Epoch AI log |
| 15 | o3-mini-2025-01-31_mediumOpenAI | closed | effort medium | 95.2%±0.5 | 31 Jan 2025 | Epoch AI log |
| 16 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 94.9%±0.5 | 20 Aug 2025 | Epoch AI log |
| 17 | o1-2024-12-17_highOpenAI | closed | effort high | 94.7%±0.6 | 13 Feb 2025 | Epoch AI log |
| 18 | o1-2024-12-17_mediumOpenAI | closed | effort medium | 94.4%±0.6 | 27 Jan 2025 | Epoch AI log |
| 19 | deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeek | open | default | 93.1%±0.7 | 31 Jan 2025 | Epoch AI log |
| 20 | claude-3-7-sonnet-20250219_64KAnthropic | closed | default | 91.2%±0.8 | 13 Mar 2025 | Epoch AI log |
| 21 | grok-3-mini-beta_lowxAI | closed | effort low | 90.9%±0.8 | 10 Apr 2025 | Epoch AI log |
| 22 | claude-3-7-sonnet-20250219_32KAnthropic | closed | default | 90.0%±0.8 | 12 Mar 2025 | Epoch AI log |
| 23 | deepseek-ai/DeepSeek-R1-Distill-Llama-70BEpoch: DeepSeek-R1-Distill-Llama-70B, DeepSeek | open | default | 89.9%±0.7 | 10 Mar 2025 | Epoch AI log |
| 24 | o1-mini-2024-09-12_highOpenAI | closed | effort high | 89.2%±0.7 | 13 Feb 2025 | Epoch AI log |
| 25 | grok-3-betaxAI | closed | default | 88.7%±0.9 | 10 Apr 2025 | Epoch AI log |
| 26 | grok-3-mini-beta_highxAI | closed | effort high | 88.1%±0.9 | 10 Apr 2025 | Epoch AI log |
| 27 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 87.3%±0.7 | 14 Apr 2025 | Epoch AI log |
| 28 | deepseek-ai/DeepSeek-R1-Distill-Qwen-14BEpoch: DeepSeek-R1-Distill-Qwen-14B, DeepSeek | open | default | 87.1%±0.7 | 11 Mar 2025 | Epoch AI log |
| 29 | claude-haiku-4-5-20251001Anthropic | closed | default | 86.9%±0.8 | 16 Oct 2025 | Epoch AI log |
| 30 | claude-3-7-sonnet-20250219_16KAnthropic | closed | default | 86.3%±0.9 | 26 Feb 2025 | Epoch AI log |
| 31 | claude-opus-4-20250514Anthropic | closed | default | 85.0%±1.0 | 22 May 2025 | Epoch AI log |
| 32 | claude-sonnet-4-20250514Anthropic | closed | default | 84.4%±1.0 | 22 May 2025 | Epoch AI log |
| 33 | o1-mini-2024-09-12_mediumOpenAI | closed | effort medium | 84.3%±0.8 | 27 Jan 2025 | Epoch AI log |
| 34 | gemini-2.0-pro-exp-02-05Google DeepMind | closed | default | 83.5%±1.0 | 6 Feb 2025 | Epoch AI log |
| 35 | gpt-4.1-2025-04-14OpenAI | closed | default | 83.0%±0.8 | 14 Apr 2025 | Epoch AI log |
| 36 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 82.2%±0.9 | 6 Feb 2025 | Epoch AI log |
| 37 | o1-preview-2024-09-12OpenAI | closed | default | 81.6%±1.1 | 27 Jan 2025 | Epoch AI log |
| 38 | mistral-medium-2505Mistral AI | closed | default | 81.6%±0.9 | 7 May 2025 | Epoch AI log |
| 39 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 78.6%±1.1 | 28 Feb 2025 | Epoch AI log |
| 40 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 75.5%±0.9 | 1 Apr 2025 | Epoch AI log |
| 41 | google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMind | open | default | 74.0%±1.1 | 13 Mar 2025 | Epoch AI log |
| 42 | meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AI | open | default | 73.0%±1.1 | 8 Apr 2025 | Epoch AI log |
| 43 | gemini-1.5-pro-002Google DeepMind | closed | default | 70.4%±1.0 | 27 Jan 2025 | Epoch AI log |
| 44 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 70.0%±1.0 | 14 Apr 2025 | Epoch AI log |
| 45 | Qwen/Qwen3-235B-A22BEpoch: qwen3-235b-a22b, Alibaba | open | default | 68.9%±0.9 | 3 Jun 2025 | Epoch AI log |
| 46 | claude-3-7-sonnet-20250219Anthropic | closed | default | 68.2%±1.1 | 24 Feb 2025 | Epoch AI log |
| 47 | qwen-max-2025-01-25Alibaba | closed | default | 67.2%±1.1 | 1 Apr 2025 | Epoch AI log |
| 48 | qwen-plus-2025-01-25Alibaba | closed | default | 65.3%±1.1 | 7 Apr 2025 | Epoch AI log |
| 49 | microsoft/phi-4Epoch: phi-4, Microsoft Research | open | default | 64.9%±1.1 | 31 Jan 2025 | Epoch AI log |
| 50 | deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeek | open | default | 64.9%±1.0 | 27 Jan 2025 | Epoch AI log |
| 51 | grok-2-1212xAI | closed | default | 63.5%±1.3 | 17 Feb 2025 | Epoch AI log |
| 52 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 63.2%±1.1 | 27 Jan 2025 | Epoch AI log |
| 53 | meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AI | open | default | 62.3%±1.2 | 8 Apr 2025 | Epoch AI log |
| 54 | gemini-1.5-flash-002Google DeepMind | closed | default | 61.9%±1.1 | 27 Jan 2025 | Epoch AI log |
| 55 | claude-3-5-sonnet-20241022Anthropic | closed | default | 56.9%±1.1 | 27 Jan 2025 | Epoch AI log |
| 56 | qwen-turbo-2024-11-01Alibaba | closed | default | 56.2%±1.1 | 7 Apr 2025 | Epoch AI log |
| 57 | Qwen/Qwen2.5-32B-InstructEpoch: qwen2.5-32b-instruct, Alibaba | open | default | 56.1%±1.1 | 30 Jan 2025 | Epoch AI log |
| 58 | gpt-4o-2024-08-06OpenAI | closed | default | 53.3%±1.1 | 27 Jan 2025 | Epoch AI log |
| 59 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 52.6%±1.1 | 27 Jan 2025 | Epoch AI log |
| 60 | claude-3-5-sonnet-20240620Anthropic | closed | default | 51.7%±1.2 | 27 Jan 2025 | Epoch AI log |
| 61 | gpt-4o-2024-05-13OpenAI | closed | default | 51.0%±1.1 | 27 Jan 2025 | Epoch AI log |
| 62 | mistral-large-2411Mistral AIno page match | open | default | 50.3%±1.1 | 25 Feb 2025 | Epoch AI log |
| 63 | gpt-4o-2024-11-20OpenAI | closed | default | 49.8%±1.1 | 5 Feb 2025 | Epoch AI log |
| 64 | meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AI | open | default | 49.8%±1.2 | 27 Jan 2025 | Epoch AI log |
| 65 | mistral-small-2503Mistral AIno page match | open | default | 46.8%±1.2 | 18 Mar 2025 | Epoch AI log |
| 66 | gpt-4-turbo-2024-04-09OpenAI | closed | default | 46.7%±1.1 | 27 Feb 2025 | Epoch AI log |
| 67 | claude-3-5-haiku-20241022Anthropic | closed | default | 46.4%±1.2 | 12 Mar 2025 | Epoch AI log |
| 68 | mistral-large-2407Mistral AIno page match | open | default | 44.8%±1.1 | 27 Jan 2025 | Epoch AI log |
| 69 | mistral-small-2501Mistral AIno page match | open | default | 44.8%±1.1 | 30 Jan 2025 | Epoch AI log |
| 70 | allenai/Llama-3.1-Tulu-3-70B-DPOEpoch: Llama-3.1-Tulu-3-70B-DPO, Allen Institute for AI,University of Washington | open | default | 42.7%±1.0 | 27 Jan 2025 | Epoch AI log |
| 71 | meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AI | open | default | 41.6%±1.2 | 27 Jan 2025 | Epoch AI log |
| 72 | gemini-1.5-pro-001Google DeepMind | closed | default | 40.7%±1.1 | 27 Jan 2025 | Epoch AI log |
| 73 | gpt-4-1106-previewOpenAI | closed | default | 40.0%±1.1 | 27 Jan 2025 | Epoch AI log |
| 74 | meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AI | open | default | 39.4%±1.0 | 27 Jan 2025 | Epoch AI log |
| 75 | Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, Alibaba | open | default | 39.1%±1.0 | 27 Jan 2025 | Epoch AI log |
| 76 | claude-3-opus-20240229Anthropic | closed | default | 37.5%±1.1 | 27 Jan 2025 | Epoch AI log |
| 77 | meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AI | open | default | 36.7%±1.0 | 27 Jan 2025 | Epoch AI log |
| 78 | gpt-4-0125-previewOpenAI | closed | default | 35.4%±1.1 | 27 Jan 2025 | Epoch AI log |
| 79 | google/gemma-2-27b-itEpoch: gemma-2-27b-it, Google DeepMind | open | default | 27.9%±1.0 | 27 Jan 2025 | Epoch AI log |
| 80 | WizardLM-2-8x22BMicrosoftno page match | open | default | 25.7%±0.9 | 27 Jan 2025 | Epoch AI log |
| 81 | 01-ai/Yi-1.5-34B-ChatEpoch: Yi-1.5-34B-Chat, 01.AI | open | default | 25.5%±0.9 | 27 Jan 2025 | Epoch AI log |
| 82 | gemini-1.5-flash-001Google DeepMind | closed | default | 25.1%±0.9 | 27 Jan 2025 | Epoch AI log |
| 83 | mistral-large-2402Mistral AI | closed | default | 24.5%±0.9 | 27 Jan 2025 | Epoch AI log |
| 84 | open-mixtral-8x22bMistral AIno page match | open | default | 24.2%±0.8 | 27 Jan 2025 | Epoch AI log |
| 85 | gpt-4-0613OpenAI | closed | default | 23.0%±0.8 | 27 Jan 2025 | Epoch AI log |
| 86 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | default | 22.9%±0.9 | 27 Jan 2025 | Epoch AI log |
| 87 | Hermes-2-Theta-Llama-3-70BNous Research,Arcee AIno page match | open | default | 22.7%±0.8 | 27 Jan 2025 | Epoch AI log |
| 88 | meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AI | open | default | 22.6%±0.8 | 27 Jan 2025 | Epoch AI log |
| 89 | google/gemma-2-9b-itEpoch: gemma-2-9b-it, Google DeepMind | open | default | 21.0%±0.9 | 27 Jan 2025 | Epoch AI log |
| 90 | claude-3-sonnet-20240229Anthropic | closed | default | 18.2%±0.8 | 27 Jan 2025 | Epoch AI log |
| 91 | microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, Microsoft | open | default | 17.6%±0.7 | 31 Jan 2025 | Epoch AI log |
| 92 | gpt-3.5-turbo-1106OpenAI | closed | default | 15.9%±0.7 | 27 Jan 2025 | Epoch AI log |
| 93 | ministral-8b-2410Mistral AIno page match | open | default | 14.9%±0.7 | 27 Jan 2025 | Epoch AI log |
| 94 | claude-3-haiku-20240307Anthropic | closed | default | 14.9%±0.7 | 27 Jan 2025 | Epoch AI log |
| 95 | ministral-3b-2410Mistral AI | closed | default | 14.4%±0.7 | 27 Jan 2025 | Epoch AI log |
| 96 | claude-2.0Anthropic | closed | default | 11.7%±0.7 | 27 Jan 2025 | Epoch AI log |
| 97 | dbrx-instructDatabricksno page match | open | default | 11.7%±0.7 | 27 Jan 2025 | Epoch AI log |
| 98 | gpt-3.5-turbo-0125OpenAI | closed | default | 11.6%±0.6 | 27 Jan 2025 | Epoch AI log |
| 99 | gemini-1.0-pro-001Google DeepMind | closed | default | 11.2%±0.6 | 27 Jan 2025 | Epoch AI log |
| 100 | open-mistral-nemo-2407Mistral AIno page match | open | default | 10.8%±0.6 | 27 Jan 2025 | Epoch AI log |
| 101 | open-mixtral-8x7bMistral AIno page match | open | default | 10.0%±0.5 | 27 Jan 2025 | Epoch AI log |
| 102 | mistralai/Mixtral-8x7B-Instruct-v0.1Epoch: Mixtral-8x7B-Instruct-v0.1, Mistral AI | open | default | 9.3%±0.5 | 27 Jan 2025 | Epoch AI log |
| 103 | deepseek-ai/deepseek-llm-67b-chatEpoch: deepseek-llm-67b-chat, DeepSeek | open | default | 6.4%±0.4 | 27 Jan 2025 | Epoch AI log |
| 104 | meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AI | open | default | 6.1%±0.4 | 27 Jan 2025 | Epoch AI log |
| 105 | 01-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AI | open | default | 5.1%±0.3 | 27 Jan 2025 | Epoch AI log |
| 106 | open-mistral-7bMistral AIno page match | open | default | 3.7%±0.3 | 27 Jan 2025 | Epoch AI log |
| 107 | mistralai/Mistral-7B-Instruct-v0.3Epoch: Mistral-7B-Instruct-v0.3, Mistral AI | open | default | 3.6%±0.3 | 27 Jan 2025 | Epoch AI log |
| 108 | meta-llama/Llama-2-70b-chat-hfEpoch: Llama-2-70b-chat-hf, Meta AI | open | default | 3.3%±0.3 | 27 Jan 2025 | Epoch AI log |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method