Skip to content
Weights Not Walls
Menu

FrontierCode

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://cognition.com/frontiercode
Score shown
Epoch's column "Main score", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
Results
46 in all: 14 for open-weight models, 31 for closed ones, 1 where Epoch does not say. 11 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K3, 44.2%, number 16 of 46 results.

FrontierCode results
#ModelWeightsSettingScoreDateSource
1claude-opus-5-5_mediumAnthropicclosedeffort medium; Harness: claude-code; Reasoning effort: medium54.6%not datedcognition.com
2claude-fable-5_unknownAnthropicclosedeffort unknown; Harness: claude-code; Reasoning effort: xhigh53.5%not datedcognition.com
3claude-opus-5_maxAnthropicclosedeffort max; Harness: claude-code; Reasoning effort: medium53.4%not datedcognition.com
4gpt-6-astra_maxOpenAIclosedeffort max; Harness: codex; Reasoning effort: max53.3%not datedcognition.com
5claude-sonnet-5-5_xhighAnthropicclosedeffort xhigh; Harness: claude-code; Reasoning effort: xhigh52.1%not datedcognition.com
6claude-fable-5-1_mediumAnthropicclosedeffort medium; Harness: claude-code; Reasoning effort: medium50.9%not datedcognition.com
7gpt-6.1-sol_mediumOpenAIclosedeffort medium; Harness: codex; Reasoning effort: medium50.2%not datedcognition.com
8swe-2_maxCognitionclosedeffort max; Harness: devin; Reasoning effort: max50.0%not datedcognition.com
9gpt-6-sol_maxOpenAIclosedeffort max; Harness: codex; Reasoning effort: max49.3%not datedcognition.com
10grok-4.6_unknownxAIclosedeffort unknown; Harness: grok-build; Reasoning effort: high48.0%not datedcognition.com
11grok-4.7_unknownxAIclosedeffort unknown; Harness: grok-build; Reasoning effort: high47.6%not datedcognition.com
12gemini-4-argon_mediumGoogle DeepMindclosedeffort medium; Harness: devin-cli; Reasoning effort: medium47.5%not datedcognition.com
13gpt-5.6-sol_unknownOpenAIclosedeffort unknown; Harness: codex; Reasoning effort: max47.5%not datedcognition.com
14claude-opus-4-8_unknownAnthropicclosedeffort unknown; Harness: claude-code; Reasoning effort: max46.5%not datedcognition.com
15claude-haiku-5-5_maxAnthropicclosedeffort max; Harness: claude-code; Reasoning effort: max46.4%not datedcognition.com
16moonshotai/Kimi-K3Epoch: kimi-k3_unknown, Moonshotopeneffort unknown; Harness: mini-swe-agent; Reasoning effort: none44.2%not datedcognition.com
17deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_high, DeepSeekopeneffort high; Harness: devin-cli; Reasoning effort: high43.9%not datedcognition.com
18gemini-3.7-flash_unknownGoogle DeepMindclosedeffort unknown; Harness: devin-cli; Reasoning effort: medium43.6%not datedcognition.com
19gpt-5.5_unknownOpenAIclosedeffort unknown; Harness: codex; Reasoning effort: xhigh43.0%not datedcognition.com
20claude-sonnet-5_unknownAnthropicclosedeffort unknown; Harness: claude-code; Reasoning effort: xhigh42.7%not datedcognition.com
21grok-4.5_unknownxAIclosedeffort unknown; Harness: grok-build; Reasoning effort: high42.4%not datedcognition.com
22gpt-6-luna_maxOpenAIclosedeffort max; Harness: codex; Reasoning effort: max42.4%not datedcognition.com
23swe-1.7Cognitionnot statedHarness: devin-cli; Reasoning effort: none42.0%not datedcognition.com
24gpt-5.6-terra_unknownOpenAIclosedeffort unknown; Harness: codex; Reasoning effort: max41.3%not datedcognition.com
25gemini-3.8-flash_mediumGoogle DeepMindclosedeffort medium; Harness: devin-cli; Reasoning effort: medium41.2%not datedcognition.com
26zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI)openeffort max; Harness: devin-cli; Reasoning effort: max40.1%not datedcognition.com
27gpt-5.6-luna_unknownOpenAIclosedeffort unknown; Harness: codex; Reasoning effort: max39.8%not datedcognition.com
28mistral-large-4-0_highMistral AIno page matchopeneffort high; Harness: chisel; Reasoning effort: high39.4%not datedcognition.com
29claude-opus-4-7_unknownAnthropicclosedeffort unknown; Harness: claude-code; Reasoning effort: max38.5%not datedcognition.com
30gemini-3.6-flash_unknownGoogle DeepMindclosedeffort unknown; Harness: devin-cli; Reasoning effort: medium34.4%not datedcognition.com
31zai-org/GLM-5.3-FlashEpoch: glm-5.3-flash_max, Z.ai (Zhipu AI)openeffort max; Harness: devin-cli; Reasoning effort: max31.8%not datedcognition.com
32moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, MoonshotopenHarness: mini-swe-agent; Reasoning effort: none30.1%not datedcognition.com
33deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_high, DeepSeekopeneffort high; Harness: devin-cli; Reasoning effort: high28.5%not datedcognition.com
34gpt-5.4-mini-2026-03-17_unknownOpenAIclosedeffort unknown; Harness: codex; Reasoning effort: xhigh27.0%not datedcognition.com
35claude-opus-4-6_unknownAnthropicclosedeffort unknown; Harness: claude-code; Reasoning effort: high26.6%not datedcognition.com
36Composer 2.5CursorclosedHarness: cursor-cli; Reasoning effort: none25.6%not datedcognition.com
37zai-org/GLM-5.2Epoch: glm-5.2_none, Z.ai (Zhipu AI)openeffort none; Harness: mini-swe-agent; Reasoning effort: none24.5%not datedcognition.com
38claude-sonnet-4-6_unknownAnthropicclosedeffort unknown; Harness: claude-code; Reasoning effort: max24.3%not datedcognition.com
39deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_unknown, DeepSeekopeneffort unknown; Harness: devin-cli; Reasoning effort: high18.8%not datedcognition.com
40deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_none, DeepSeekopeneffort none; Harness: mini-swe-agent; Reasoning effort: none17.6%not datedcognition.com
41MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMaxopenHarness: mini-swe-agent; Reasoning effort: none14.7%not datedcognition.com
42thinkingmachines/InklingEpoch: Inkling, Thinking MachinesopenHarness: mini-swe-agent; Reasoning effort: 0.9914.0%not datedcognition.com
43nemotron-3-ultraNvidiano page matchopenHarness: devin-cli; Reasoning effort: none13.6%not datedcognition.com
44qwen3.7-plusAlibabaclosedHarness: mini-swe-agent; Reasoning effort: none10.2%not datedcognition.com
45swe-1.6CognitionclosedHarness: devin-cli; Reasoning effort: none9.4%not datedcognition.com
46mistral-medium-2604Mistral AIno page matchopenHarness: devin-cli; Reasoning effort: none8.0%not datedcognition.com

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method