Skip to content
Weights Not Walls
Menu

FrontierMath-2025-02-28-Private

Re-run by Epoch AI

Unpublished research-level mathematics problems written by mathematicians for Epoch AI, with answers that can be checked automatically.

Kind
Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
Score shown
Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
Results
101 in all: 13 for open-weight models, 84 for closed ones, 4 where Epoch does not say. 12 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K2.6, 39.0%, number 15 of 101 results.

FrontierMath-2025-02-28-Private results
#ModelWeightsSettingScoreDateSource
1gpt-5.5-pro-pre-release_highOpenAIclosedeffort high52.4%±2.923 Apr 2026Epoch AI
2gpt-5.5-pre-release_xhighOpenAIclosedeffort xhigh51.7%±2.923 Apr 2026Epoch AI
3gpt-5.5-pro-pre-release_xhighOpenAIclosedeffort xhigh51.0%±2.923 Apr 2026Epoch AI
4gpt-5.4-pro-2026-03-05_xhighOpenAIclosedeffort xhigh50.0%±2.96 Mar 2026Epoch AI
5gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh47.6%±2.96 Mar 2026Epoch AI
6claude-opus-4-8_maxAnthropicclosedeffort max47.2%±2.98 Jun 2026Epoch AI log
7claude-opus-4-7_xhighAnthropicclosedeffort xhigh43.8%±2.917 Apr 2026Epoch AI
8claude-opus-4-6_maxAnthropicclosedeffort max40.7%±2.912 Feb 2026Epoch AI log
9gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh40.7%±2.913 Dec 2025Epoch AI
10gpt-5.2-2025-12-11_highOpenAIclosedeffort high40.3%±2.911 Dec 2025Epoch AI log
11claude-opus-4-6_32KAnthropiccloseddefault40.0%±2.96 Feb 2026Epoch AI log
12claude-opus-4-6_64KAnthropiccloseddefault39.7%±2.96 Feb 2026Epoch AI log
13muse-sparkMeta AIcloseddefault39.0%±2.98 Apr 2026Epoch AI
14gemini-3.5-flash_highGoogle DeepMindclosedeffort high39.0%±2.922 May 2026Epoch AI
15moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault39.0%±2.97 May 2026Epoch AI
16claude-opus-4-6Anthropiccloseddefault38.3%±2.95 Feb 2026Epoch AI log
17gemini-3-pro-previewGoogle DeepMindcloseddefault37.6%±2.821 Nov 2025Epoch AI
18gemini-3.1-pro-previewGoogle DeepMindcloseddefault36.9%±2.819 Feb 2026Epoch AI log
19gpt-5.2-2025-12-11_mediumOpenAIclosedeffort medium36.9%±2.811 Dec 2025Epoch AI log
20gemini-3-flash-previewGoogle DeepMindcloseddefault35.6%±2.817 Dec 2025Epoch AI log
21zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI)opendefault33.4%±2.811 May 2026Epoch AI
22gpt-5-2025-08-07_highOpenAIclosedeffort high32.4%±2.813 Nov 2025Epoch AI log
23claude-sonnet-4-6_16KAnthropiccloseddefault32.4%±2.820 Feb 2026Epoch AI log
24gpt-5.1-2025-11-13_highOpenAIclosedeffort high31.0%±2.713 Nov 2025Epoch AI log
25gemini-2.5-deep-think-2025-08-01-webappGoogle,Google DeepMindcloseddefault29.0%±2.7not datedEpoch AI
26gpt-5.4-mini-2026-03-17_highOpenAIclosedeffort high28.3%±2.115 Apr 2026Epoch AI
27moonshotai/Kimi-K2.5Epoch: fireworks/kimi-k2p5, Moonshotopendefault27.9%±2.63 Feb 2026Epoch AI
28gpt-5-2025-08-07_mediumOpenAIclosedeffort medium27.2%±2.613 Nov 2025Epoch AI log
29gpt-5-mini-2025-08-07_highOpenAIclosedeffort high27.2%±2.613 Nov 2025Epoch AI log
30gpt-5.1-2025-11-13_mediumOpenAIclosedeffort medium26.9%±2.617 Nov 2025Epoch AI log
31gpt-5.2-2025-12-11_lowOpenAIclosedeffort low26.6%±2.611 Dec 2025Epoch AI log
32qwen3.6-plusAlibabacloseddefault26.2%±2.612 May 2026Epoch AI
33gpt-5.4-nano-2026-03-17_highOpenAIclosedeffort high25.9%±2.615 Apr 2026Epoch AI
34o4-mini-2025-04-16_highOpenAIclosedeffort high24.8%±2.513 Nov 2025Epoch AI log
35qwen3.6-max-previewAlibabanot stateddefault23.1%±2.527 May 2026Epoch AI
36deepseek-ai/DeepSeek-V3.2Epoch: fireworks/deepseek-v3p2, DeepSeekopendefault22.1%±4.122 Dec 2025Epoch AI
37moonshotai/Kimi-K2-ThinkingEpoch: moonshotai/Kimi-K2-Thinking, Moonshotopendefault21.4%±2.45 Dec 2025Epoch AI log
38qwen3.5-plusAlibabanot stateddefault21.0%±2.414 May 2026Epoch AI
39claude-opus-4-5-20251101Anthropiccloseddefault20.7%±2.425 Nov 2025Epoch AI log
40claude-opus-4-5-20251101_32KAnthropiccloseddefault20.7%±2.425 Nov 2025Epoch AI log
41claude-opus-4-5-20251101_16KAnthropiccloseddefault20.3%±2.425 Nov 2025Epoch AI log
42gpt-5-mini-2025-08-07_mediumOpenAIclosedeffort medium20.3%±2.413 Nov 2025Epoch AI log
43grok-4-0709xAIcloseddefault19.7%±2.313 Nov 2025Epoch AI log
44o4-mini-2025-04-16_mediumOpenAIclosedeffort medium19.0%±2.313 Nov 2025Epoch AI log
45o3-2025-04-16_highOpenAIclosedeffort high18.7%±2.316 Nov 2025Epoch AI log
46gpt-5.1-2025-11-13_lowOpenAIclosedeffort low17.3%±2.225 Nov 2025Epoch AI log
47o3-2025-04-16_mediumOpenAIclosedeffort medium16.9%±2.216 Nov 2025Epoch AI log
48zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI)opendefault16.4%±2.219 Feb 2026Epoch AI log
49claude-sonnet-4-5-20250929_32KAnthropiccloseddefault15.2%±2.116 Nov 2025Epoch AI log
50gemini-2.5-proGoogle DeepMindcloseddefault14.1%±2.024 Nov 2025Epoch AI log
51claude-sonnet-4-5-20250929_59KAnthropiccloseddefault13.5%±2.013 Nov 2025Epoch AI log
52o3-mini-2025-01-31_highOpenAIclosedeffort high12.4%±1.916 Nov 2025Epoch AI log
53o4-mini-2025-04-16_lowOpenAIclosedeffort low10.7%±1.816 Nov 2025Epoch AI log
54gemini-2.5-pro-preview-06-05Google DeepMindcloseddefault10.3%±1.83 Jul 2025Epoch AI log
55qwen3.6-flashAlibabanot stateddefault10.3%±1.812 May 2026Epoch AI
56o3-2025-04-16_lowOpenAIclosedeffort low9.7%±1.717 Nov 2025Epoch AI log
57claude-sonnet-4-5-20250929Anthropiccloseddefault9.3%±1.716 Nov 2025Epoch AI log
58o1-2024-12-17_highOpenAIclosedeffort high9.3%±1.77 Mar 2025Epoch AI log
59Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen/Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault8.5%±1.711 Dec 2025Epoch AI log
60gpt-5-nano-2025-08-07_highOpenAIclosedeffort high8.3%±1.630 Oct 2025Epoch AI log
61o3-mini-2025-01-31_mediumOpenAIclosedeffort medium8.1%±1.36 Mar 2025Epoch AI log
62claude-opus-4-1-20250805_27KAnthropiccloseddefault7.2%±1.55 Aug 2025Epoch AI log
63gpt-5-nano-2025-08-07_mediumOpenAIclosedeffort medium7.2%±1.57 Aug 2025Epoch AI log
64qwen3.5-flashAlibabanot stateddefault6.2%±1.412 May 2026Epoch AI
65claude-haiku-4-5-20251001_32KAnthropiccloseddefault5.9%±1.422 Oct 2025Epoch AI log
66claude-opus-4-1-20250805Anthropiccloseddefault5.9%±1.45 Aug 2025Epoch AI log
67grok-3-mini-beta_highxAIclosedeffort high5.9%±1.410 Apr 2025Epoch AI log
68gpt-4.1-2025-04-14OpenAIcloseddefault5.5%±1.314 Apr 2025Epoch AI log
69gemini-2.5-flashGoogle DeepMindcloseddefault4.8%±1.318 Dec 2025Epoch AI log
70claude-opus-4-20250514Anthropiccloseddefault4.5%±1.24 Jul 2025Epoch AI log
71gpt-4.1-mini-2025-04-14OpenAIcloseddefault4.5%±1.214 Apr 2025Epoch AI log
72claude-3-7-sonnet-20250219_16KAnthropiccloseddefault4.1%±1.26 Mar 2025Epoch AI log
73claude-haiku-4-5-20251001Anthropiccloseddefault4.1%±1.216 Oct 2025Epoch AI log
74claude-opus-4-20250514_27KAnthropiccloseddefault4.1%±1.25 Aug 2025Epoch AI log
75claude-sonnet-4-20250514Anthropiccloseddefault4.1%±1.24 Jul 2025Epoch AI log
76zai-org/GLM-4.6Epoch: zai-org/GLM-4.6, Z.ai (Zhipu AI),Tsinghua Universityopendefault3.8%±1.18 Dec 2025Epoch AI log
77grok-3-betaxAIcloseddefault3.8%±1.110 Apr 2025Epoch AI log
78claude-3-7-sonnet-20250219_32KAnthropiccloseddefault3.4%±1.113 Mar 2025Epoch AI log
79claude-3-7-sonnet-20250219Anthropiccloseddefault3.1%±1.06 Mar 2025Epoch AI log
80claude-3-7-sonnet-20250219_64KAnthropiccloseddefault3.1%±1.013 Mar 2025Epoch AI log
81grok-3-mini-beta_lowxAIclosedeffort low2.8%±1.010 Apr 2025Epoch AI log
82zai-org/GLM-4.7Epoch: zai-org/GLM-4.7, Z.ai (Zhipu AI)opendefault2.4%±0.930 Jan 2026Epoch AI log
83gpt-5.1-2025-11-13_noneOpenAIclosedeffort none2.1%±0.825 Nov 2025Epoch AI log
84claude-3-5-sonnet-20241022Anthropiccloseddefault2.1%±0.86 Mar 2025Epoch AI log
85deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopendefault1.7%±0.87 Mar 2025Epoch AI log
86gemini-2.0-flash-001Google DeepMind,Googlecloseddefault1.7%±0.89 Mar 2025Epoch AI log
87o1-mini-2024-09-12_mediumOpenAIclosedeffort medium1.7%±0.86 Mar 2025Epoch AI log
88qwen-plus-2025-04-28Alibabacloseddefault1.7%±0.814 May 2025Epoch AI log
89o1-mini-2024-09-12_highOpenAIclosedeffort high1.4%±0.76 Mar 2025Epoch AI log
90claude-3-5-sonnet-20240620Anthropiccloseddefault1.0%±0.67 Mar 2025Epoch AI log
91gpt-4.1-nano-2025-04-14OpenAIcloseddefault1.0%±0.614 Apr 2025Epoch AI log
92qwen-max-2025-01-25Alibabacloseddefault1.0%±0.62 Apr 2025Epoch AI log
93grok-2-1212xAIcloseddefault0.7%±0.56 Mar 2025Epoch AI log
94meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AIopendefault0.7%±0.58 Apr 2025Epoch AI log
95mistral-medium-2505Mistral AIcloseddefault0.3%±0.37 May 2025Epoch AI log
96claude-3-5-haiku-20241022Anthropiccloseddefault0.3%±0.37 Mar 2025Epoch AI log
97gpt-4o-2024-08-06OpenAIcloseddefault0.3%±0.37 Mar 2025Epoch AI log
98gpt-4o-2024-11-20OpenAIcloseddefault0.3%±0.36 Mar 2025Epoch AI log
99mistral-large-2411Mistral AIno page matchopendefault0.3%±0.36 Mar 2025Epoch AI log
100gemini-1.5-flash-002Google DeepMindcloseddefault0.0%7 Mar 2025Epoch AI log
101meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AIopendefault0.0%8 Apr 2025Epoch AI log

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method