FrontierMath-2025-02-28-Private
Re-run by Epoch AIUnpublished research-level mathematics problems written by mathematicians for Epoch AI, with answers that can be checked automatically.
- Kind
- Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
- Score shown
- Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
- Results
- 101 in all: 13 for open-weight models, 84 for closed ones, 4 where Epoch does not say. 12 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K2.6, 39.0%, number 15 of 101 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | gpt-5.5-pro-pre-release_highOpenAI | closed | effort high | 52.4%±2.9 | 23 Apr 2026 | Epoch AI |
| 2 | gpt-5.5-pre-release_xhighOpenAI | closed | effort xhigh | 51.7%±2.9 | 23 Apr 2026 | Epoch AI |
| 3 | gpt-5.5-pro-pre-release_xhighOpenAI | closed | effort xhigh | 51.0%±2.9 | 23 Apr 2026 | Epoch AI |
| 4 | gpt-5.4-pro-2026-03-05_xhighOpenAI | closed | effort xhigh | 50.0%±2.9 | 6 Mar 2026 | Epoch AI |
| 5 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 47.6%±2.9 | 6 Mar 2026 | Epoch AI |
| 6 | claude-opus-4-8_maxAnthropic | closed | effort max | 47.2%±2.9 | 8 Jun 2026 | Epoch AI log |
| 7 | claude-opus-4-7_xhighAnthropic | closed | effort xhigh | 43.8%±2.9 | 17 Apr 2026 | Epoch AI |
| 8 | claude-opus-4-6_maxAnthropic | closed | effort max | 40.7%±2.9 | 12 Feb 2026 | Epoch AI log |
| 9 | gpt-5.2-2025-12-11_xhighOpenAI | closed | effort xhigh | 40.7%±2.9 | 13 Dec 2025 | Epoch AI |
| 10 | gpt-5.2-2025-12-11_highOpenAI | closed | effort high | 40.3%±2.9 | 11 Dec 2025 | Epoch AI log |
| 11 | claude-opus-4-6_32KAnthropic | closed | default | 40.0%±2.9 | 6 Feb 2026 | Epoch AI log |
| 12 | claude-opus-4-6_64KAnthropic | closed | default | 39.7%±2.9 | 6 Feb 2026 | Epoch AI log |
| 13 | muse-sparkMeta AI | closed | default | 39.0%±2.9 | 8 Apr 2026 | Epoch AI |
| 14 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 39.0%±2.9 | 22 May 2026 | Epoch AI |
| 15 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 39.0%±2.9 | 7 May 2026 | Epoch AI |
| 16 | claude-opus-4-6Anthropic | closed | default | 38.3%±2.9 | 5 Feb 2026 | Epoch AI log |
| 17 | gemini-3-pro-previewGoogle DeepMind | closed | default | 37.6%±2.8 | 21 Nov 2025 | Epoch AI |
| 18 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 36.9%±2.8 | 19 Feb 2026 | Epoch AI log |
| 19 | gpt-5.2-2025-12-11_mediumOpenAI | closed | effort medium | 36.9%±2.8 | 11 Dec 2025 | Epoch AI log |
| 20 | gemini-3-flash-previewGoogle DeepMind | closed | default | 35.6%±2.8 | 17 Dec 2025 | Epoch AI log |
| 21 | zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI) | open | default | 33.4%±2.8 | 11 May 2026 | Epoch AI |
| 22 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 32.4%±2.8 | 13 Nov 2025 | Epoch AI log |
| 23 | claude-sonnet-4-6_16KAnthropic | closed | default | 32.4%±2.8 | 20 Feb 2026 | Epoch AI log |
| 24 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 31.0%±2.7 | 13 Nov 2025 | Epoch AI log |
| 25 | gemini-2.5-deep-think-2025-08-01-webappGoogle,Google DeepMind | closed | default | 29.0%±2.7 | not dated | Epoch AI |
| 26 | gpt-5.4-mini-2026-03-17_highOpenAI | closed | effort high | 28.3%±2.1 | 15 Apr 2026 | Epoch AI |
| 27 | moonshotai/Kimi-K2.5Epoch: fireworks/kimi-k2p5, Moonshot | open | default | 27.9%±2.6 | 3 Feb 2026 | Epoch AI |
| 28 | gpt-5-2025-08-07_mediumOpenAI | closed | effort medium | 27.2%±2.6 | 13 Nov 2025 | Epoch AI log |
| 29 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 27.2%±2.6 | 13 Nov 2025 | Epoch AI log |
| 30 | gpt-5.1-2025-11-13_mediumOpenAI | closed | effort medium | 26.9%±2.6 | 17 Nov 2025 | Epoch AI log |
| 31 | gpt-5.2-2025-12-11_lowOpenAI | closed | effort low | 26.6%±2.6 | 11 Dec 2025 | Epoch AI log |
| 32 | qwen3.6-plusAlibaba | closed | default | 26.2%±2.6 | 12 May 2026 | Epoch AI |
| 33 | gpt-5.4-nano-2026-03-17_highOpenAI | closed | effort high | 25.9%±2.6 | 15 Apr 2026 | Epoch AI |
| 34 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 24.8%±2.5 | 13 Nov 2025 | Epoch AI log |
| 35 | qwen3.6-max-previewAlibaba | not stated | default | 23.1%±2.5 | 27 May 2026 | Epoch AI |
| 36 | deepseek-ai/DeepSeek-V3.2Epoch: fireworks/deepseek-v3p2, DeepSeek | open | default | 22.1%±4.1 | 22 Dec 2025 | Epoch AI |
| 37 | moonshotai/Kimi-K2-ThinkingEpoch: moonshotai/Kimi-K2-Thinking, Moonshot | open | default | 21.4%±2.4 | 5 Dec 2025 | Epoch AI log |
| 38 | qwen3.5-plusAlibaba | not stated | default | 21.0%±2.4 | 14 May 2026 | Epoch AI |
| 39 | claude-opus-4-5-20251101Anthropic | closed | default | 20.7%±2.4 | 25 Nov 2025 | Epoch AI log |
| 40 | claude-opus-4-5-20251101_32KAnthropic | closed | default | 20.7%±2.4 | 25 Nov 2025 | Epoch AI log |
| 41 | claude-opus-4-5-20251101_16KAnthropic | closed | default | 20.3%±2.4 | 25 Nov 2025 | Epoch AI log |
| 42 | gpt-5-mini-2025-08-07_mediumOpenAI | closed | effort medium | 20.3%±2.4 | 13 Nov 2025 | Epoch AI log |
| 43 | grok-4-0709xAI | closed | default | 19.7%±2.3 | 13 Nov 2025 | Epoch AI log |
| 44 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 19.0%±2.3 | 13 Nov 2025 | Epoch AI log |
| 45 | o3-2025-04-16_highOpenAI | closed | effort high | 18.7%±2.3 | 16 Nov 2025 | Epoch AI log |
| 46 | gpt-5.1-2025-11-13_lowOpenAI | closed | effort low | 17.3%±2.2 | 25 Nov 2025 | Epoch AI log |
| 47 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 16.9%±2.2 | 16 Nov 2025 | Epoch AI log |
| 48 | zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI) | open | default | 16.4%±2.2 | 19 Feb 2026 | Epoch AI log |
| 49 | claude-sonnet-4-5-20250929_32KAnthropic | closed | default | 15.2%±2.1 | 16 Nov 2025 | Epoch AI log |
| 50 | gemini-2.5-proGoogle DeepMind | closed | default | 14.1%±2.0 | 24 Nov 2025 | Epoch AI log |
| 51 | claude-sonnet-4-5-20250929_59KAnthropic | closed | default | 13.5%±2.0 | 13 Nov 2025 | Epoch AI log |
| 52 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 12.4%±1.9 | 16 Nov 2025 | Epoch AI log |
| 53 | o4-mini-2025-04-16_lowOpenAI | closed | effort low | 10.7%±1.8 | 16 Nov 2025 | Epoch AI log |
| 54 | gemini-2.5-pro-preview-06-05Google DeepMind | closed | default | 10.3%±1.8 | 3 Jul 2025 | Epoch AI log |
| 55 | qwen3.6-flashAlibaba | not stated | default | 10.3%±1.8 | 12 May 2026 | Epoch AI |
| 56 | o3-2025-04-16_lowOpenAI | closed | effort low | 9.7%±1.7 | 17 Nov 2025 | Epoch AI log |
| 57 | claude-sonnet-4-5-20250929Anthropic | closed | default | 9.3%±1.7 | 16 Nov 2025 | Epoch AI log |
| 58 | o1-2024-12-17_highOpenAI | closed | effort high | 9.3%±1.7 | 7 Mar 2025 | Epoch AI log |
| 59 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen/Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 8.5%±1.7 | 11 Dec 2025 | Epoch AI log |
| 60 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 8.3%±1.6 | 30 Oct 2025 | Epoch AI log |
| 61 | o3-mini-2025-01-31_mediumOpenAI | closed | effort medium | 8.1%±1.3 | 6 Mar 2025 | Epoch AI log |
| 62 | claude-opus-4-1-20250805_27KAnthropic | closed | default | 7.2%±1.5 | 5 Aug 2025 | Epoch AI log |
| 63 | gpt-5-nano-2025-08-07_mediumOpenAI | closed | effort medium | 7.2%±1.5 | 7 Aug 2025 | Epoch AI log |
| 64 | qwen3.5-flashAlibaba | not stated | default | 6.2%±1.4 | 12 May 2026 | Epoch AI |
| 65 | claude-haiku-4-5-20251001_32KAnthropic | closed | default | 5.9%±1.4 | 22 Oct 2025 | Epoch AI log |
| 66 | claude-opus-4-1-20250805Anthropic | closed | default | 5.9%±1.4 | 5 Aug 2025 | Epoch AI log |
| 67 | grok-3-mini-beta_highxAI | closed | effort high | 5.9%±1.4 | 10 Apr 2025 | Epoch AI log |
| 68 | gpt-4.1-2025-04-14OpenAI | closed | default | 5.5%±1.3 | 14 Apr 2025 | Epoch AI log |
| 69 | gemini-2.5-flashGoogle DeepMind | closed | default | 4.8%±1.3 | 18 Dec 2025 | Epoch AI log |
| 70 | claude-opus-4-20250514Anthropic | closed | default | 4.5%±1.2 | 4 Jul 2025 | Epoch AI log |
| 71 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 4.5%±1.2 | 14 Apr 2025 | Epoch AI log |
| 72 | claude-3-7-sonnet-20250219_16KAnthropic | closed | default | 4.1%±1.2 | 6 Mar 2025 | Epoch AI log |
| 73 | claude-haiku-4-5-20251001Anthropic | closed | default | 4.1%±1.2 | 16 Oct 2025 | Epoch AI log |
| 74 | claude-opus-4-20250514_27KAnthropic | closed | default | 4.1%±1.2 | 5 Aug 2025 | Epoch AI log |
| 75 | claude-sonnet-4-20250514Anthropic | closed | default | 4.1%±1.2 | 4 Jul 2025 | Epoch AI log |
| 76 | zai-org/GLM-4.6Epoch: zai-org/GLM-4.6, Z.ai (Zhipu AI),Tsinghua University | open | default | 3.8%±1.1 | 8 Dec 2025 | Epoch AI log |
| 77 | grok-3-betaxAI | closed | default | 3.8%±1.1 | 10 Apr 2025 | Epoch AI log |
| 78 | claude-3-7-sonnet-20250219_32KAnthropic | closed | default | 3.4%±1.1 | 13 Mar 2025 | Epoch AI log |
| 79 | claude-3-7-sonnet-20250219Anthropic | closed | default | 3.1%±1.0 | 6 Mar 2025 | Epoch AI log |
| 80 | claude-3-7-sonnet-20250219_64KAnthropic | closed | default | 3.1%±1.0 | 13 Mar 2025 | Epoch AI log |
| 81 | grok-3-mini-beta_lowxAI | closed | effort low | 2.8%±1.0 | 10 Apr 2025 | Epoch AI log |
| 82 | zai-org/GLM-4.7Epoch: zai-org/GLM-4.7, Z.ai (Zhipu AI) | open | default | 2.4%±0.9 | 30 Jan 2026 | Epoch AI log |
| 83 | gpt-5.1-2025-11-13_noneOpenAI | closed | effort none | 2.1%±0.8 | 25 Nov 2025 | Epoch AI log |
| 84 | claude-3-5-sonnet-20241022Anthropic | closed | default | 2.1%±0.8 | 6 Mar 2025 | Epoch AI log |
| 85 | deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeek | open | default | 1.7%±0.8 | 7 Mar 2025 | Epoch AI log |
| 86 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 1.7%±0.8 | 9 Mar 2025 | Epoch AI log |
| 87 | o1-mini-2024-09-12_mediumOpenAI | closed | effort medium | 1.7%±0.8 | 6 Mar 2025 | Epoch AI log |
| 88 | qwen-plus-2025-04-28Alibaba | closed | default | 1.7%±0.8 | 14 May 2025 | Epoch AI log |
| 89 | o1-mini-2024-09-12_highOpenAI | closed | effort high | 1.4%±0.7 | 6 Mar 2025 | Epoch AI log |
| 90 | claude-3-5-sonnet-20240620Anthropic | closed | default | 1.0%±0.6 | 7 Mar 2025 | Epoch AI log |
| 91 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 1.0%±0.6 | 14 Apr 2025 | Epoch AI log |
| 92 | qwen-max-2025-01-25Alibaba | closed | default | 1.0%±0.6 | 2 Apr 2025 | Epoch AI log |
| 93 | grok-2-1212xAI | closed | default | 0.7%±0.5 | 6 Mar 2025 | Epoch AI log |
| 94 | meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AI | open | default | 0.7%±0.5 | 8 Apr 2025 | Epoch AI log |
| 95 | mistral-medium-2505Mistral AI | closed | default | 0.3%±0.3 | 7 May 2025 | Epoch AI log |
| 96 | claude-3-5-haiku-20241022Anthropic | closed | default | 0.3%±0.3 | 7 Mar 2025 | Epoch AI log |
| 97 | gpt-4o-2024-08-06OpenAI | closed | default | 0.3%±0.3 | 7 Mar 2025 | Epoch AI log |
| 98 | gpt-4o-2024-11-20OpenAI | closed | default | 0.3%±0.3 | 6 Mar 2025 | Epoch AI log |
| 99 | mistral-large-2411Mistral AIno page match | open | default | 0.3%±0.3 | 6 Mar 2025 | Epoch AI log |
| 100 | gemini-1.5-flash-002Google DeepMind | closed | default | 0.0% | 7 Mar 2025 | Epoch AI log |
| 101 | meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AI | open | default | 0.0% | 8 Apr 2025 | Epoch AI log |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method