Skip to content
Weights Not Walls
Menu

EnigmaEval

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://labs.scale.com/leaderboard/enigma_eval
Score shown
Epoch's column "Accuracy", as a percentage, higher is better. Error bars as the source gives them.
Results
46 in all: 4 for open-weight models, 42 for closed ones. 3 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K2.5, 3.4%, number 29 of 46 results.

EnigmaEval results
#ModelWeightsSettingScoreDateSource
1claude-fable-5_highAnthropicclosedeffort high39.3%36.5 to 42.1not datedlabs.scale.com
2gpt-5.6-sol_highOpenAIclosedeffort high37.1%34.3 to 39.9not datedlabs.scale.com
3gemini-3.1-pro-previewGoogle DeepMindcloseddefault36.8%34.1 to 39.5not datedlabs.scale.com
4gemini-3.5-flash_highGoogle DeepMindclosedeffort high25.4%23.0 to 27.9not datedlabs.scale.com
5gpt-5.4-pro-2026-03-05_unknownOpenAIclosedeffort unknown23.8%21.4 to 26.3not datedlabs.scale.com
6claude-opus-4-8_xhighAnthropicclosedeffort xhigh23.5%21.1 to 25.9not datedlabs.scale.com
7gpt-5-pro-2025-10-06_unknownOpenAIclosedeffort unknown18.8%16.5 to 21.0not datedlabs.scale.com
8gemini-3-pro-previewGoogle DeepMindcloseddefault18.2%16.0 to 20.4not datedlabs.scale.com
9gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh16.0%13.9 to 18.1not datedlabs.scale.com
10o3-2025-04-16_mediumOpenAIclosedeffort medium13.1%11.2 to 15.0not datedlabs.scale.com
11claude-opus-4-5-20251101_unknownAnthropicclosedeffort unknown11.9%10.1 to 13.8not datedlabs.scale.com
12o3-2025-04-16_highOpenAIclosedeffort high11.9%10.1 to 13.8not datedlabs.scale.com
13gpt-5.1-2025-11-13_unknownOpenAIclosedeffort unknown11.2%9.4 to 13.0not datedlabs.scale.com
14gpt-5-2025-08-07_unknownOpenAIclosedeffort unknown10.5%8.7 to 12.2not datedlabs.scale.com
15gpt-5.2-2025-12-11_unknownOpenAIclosedeffort unknown10.4%8.6 to 12.1not datedlabs.scale.com
16o4-mini-2025-04-16_highOpenAIclosedeffort high9.2%7.6 to 10.9not datedlabs.scale.com
17gpt-5-mini-2025-08-07_unknownOpenAIclosedeffort unknown8.2%6.6 to 9.8not datedlabs.scale.com
18claude-opus-4-6_maxAnthropicclosedeffort max7.6%6.1 to 9.1not datedlabs.scale.com
19claude-opus-4-1-20250805_unknownAnthropicclosedeffort unknown7.2%5.7 to 8.6not datedlabs.scale.com
20claude-opus-4-6Anthropiccloseddefault6.8%5.4 to 8.3not datedlabs.scale.com
21o4-mini-2025-04-16_mediumOpenAIclosedeffort medium6.8%6.0 to 7.6not datedlabs.scale.com
22o1-pro-2025-03-19OpenAIcloseddefault6.1%4.8 to 7.5not datedlabs.scale.com
23claude-sonnet-4-5-20250929_unknownAnthropicclosedeffort unknown6.0%4.6 to 7.3not datedlabs.scale.com
24o1-2024-12-17_unknownOpenAIclosedeffort unknown5.7%4.3 to 7.0not datedlabs.scale.com
25claude-opus-4-20250514_unknownAnthropicclosedeffort unknown5.6%4.3 to 6.9not datedlabs.scale.com
26gemini-2.5-pro-preview-06-05Google DeepMindcloseddefault5.6%4.3 to 6.9not datedlabs.scale.com
27claude-3-7-sonnet-20250219_unknownAnthropicclosedeffort unknown4.2%3.1 to 5.4not datedlabs.scale.com
28gemini-2.5-pro-exp-03-25Google DeepMindcloseddefault4.1%3.0 to 5.3not datedlabs.scale.com
29moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshotopendefault3.4%2.4 to 4.4not datedlabs.scale.com
30gpt-4.5-preview-2025-02-27OpenAIcloseddefault3.2%2.2 to 4.2not datedlabs.scale.com
31claude-sonnet-4-20250514_unknownAnthropicclosedeffort unknown3.1%2.1 to 4.1not datedlabs.scale.com
32gemini-3.1-flash-liteGooglecloseddefault3.0%2.1 to 4.0not datedlabs.scale.com
33gemini-2.5-flash-preview-05-20Google DeepMindcloseddefault2.7%1.8 to 3.6not datedlabs.scale.com
34gemini-2.5-pro-preview-05-06Google DeepMindcloseddefault2.4%1.5 to 3.2not datedlabs.scale.com
35claude-3-7-sonnet-20250219Anthropiccloseddefault2.3%1.4 to 3.1not datedlabs.scale.com
36gpt-4.1-2025-04-14OpenAIcloseddefault2.2%1.3 to 3.0not datedlabs.scale.com
37gpt-5.1-2025-11-13_noneOpenAIclosedeffort none1.9%1.1 to 2.7not datedlabs.scale.com
38gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Googlecloseddefault1.1%0.5 to 1.7not datedlabs.scale.com
39claude-3-5-sonnet-20241022Anthropiccloseddefault0.9%0.4 to 1.5not datedlabs.scale.com
40pixtral-large-2411Mistral AIno page matchopendefault0.8%0.3 to 1.4not datedlabs.scale.com
41claude-3-opus-20240229Anthropiccloseddefault0.8%0.4 to 1.3not datedlabs.scale.com
42gpt-4o-2024-11-20OpenAIcloseddefault0.8%0.4 to 1.2not datedlabs.scale.com
43gemini-2.0-pro-exp-02-05Google DeepMindcloseddefault0.7%0.2 to 1.2not datedlabs.scale.com
44gemini-2.0-flash-02-05Google DeepMind,Googlecloseddefault0.6%0.2 to 1.1not datedlabs.scale.com
45meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AIopendefault0.6%0.1 to 1.0not datedlabs.scale.com
46meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AIopendefault0.4%0.0 to 0.7not datedlabs.scale.com

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method