EnigmaEval
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://labs.scale.com/leaderboard/enigma_eval
- Score shown
- Epoch's column "Accuracy", as a percentage, higher is better. Error bars as the source gives them.
- Results
- 46 in all: 4 for open-weight models, 42 for closed ones. 3 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K2.5, 3.4%, number 29 of 46 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-fable-5_highAnthropic | closed | effort high | 39.3%36.5 to 42.1 | not dated | labs.scale.com |
| 2 | gpt-5.6-sol_highOpenAI | closed | effort high | 37.1%34.3 to 39.9 | not dated | labs.scale.com |
| 3 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 36.8%34.1 to 39.5 | not dated | labs.scale.com |
| 4 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 25.4%23.0 to 27.9 | not dated | labs.scale.com |
| 5 | gpt-5.4-pro-2026-03-05_unknownOpenAI | closed | effort unknown | 23.8%21.4 to 26.3 | not dated | labs.scale.com |
| 6 | claude-opus-4-8_xhighAnthropic | closed | effort xhigh | 23.5%21.1 to 25.9 | not dated | labs.scale.com |
| 7 | gpt-5-pro-2025-10-06_unknownOpenAI | closed | effort unknown | 18.8%16.5 to 21.0 | not dated | labs.scale.com |
| 8 | gemini-3-pro-previewGoogle DeepMind | closed | default | 18.2%16.0 to 20.4 | not dated | labs.scale.com |
| 9 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 16.0%13.9 to 18.1 | not dated | labs.scale.com |
| 10 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 13.1%11.2 to 15.0 | not dated | labs.scale.com |
| 11 | claude-opus-4-5-20251101_unknownAnthropic | closed | effort unknown | 11.9%10.1 to 13.8 | not dated | labs.scale.com |
| 12 | o3-2025-04-16_highOpenAI | closed | effort high | 11.9%10.1 to 13.8 | not dated | labs.scale.com |
| 13 | gpt-5.1-2025-11-13_unknownOpenAI | closed | effort unknown | 11.2%9.4 to 13.0 | not dated | labs.scale.com |
| 14 | gpt-5-2025-08-07_unknownOpenAI | closed | effort unknown | 10.5%8.7 to 12.2 | not dated | labs.scale.com |
| 15 | gpt-5.2-2025-12-11_unknownOpenAI | closed | effort unknown | 10.4%8.6 to 12.1 | not dated | labs.scale.com |
| 16 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 9.2%7.6 to 10.9 | not dated | labs.scale.com |
| 17 | gpt-5-mini-2025-08-07_unknownOpenAI | closed | effort unknown | 8.2%6.6 to 9.8 | not dated | labs.scale.com |
| 18 | claude-opus-4-6_maxAnthropic | closed | effort max | 7.6%6.1 to 9.1 | not dated | labs.scale.com |
| 19 | claude-opus-4-1-20250805_unknownAnthropic | closed | effort unknown | 7.2%5.7 to 8.6 | not dated | labs.scale.com |
| 20 | claude-opus-4-6Anthropic | closed | default | 6.8%5.4 to 8.3 | not dated | labs.scale.com |
| 21 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 6.8%6.0 to 7.6 | not dated | labs.scale.com |
| 22 | o1-pro-2025-03-19OpenAI | closed | default | 6.1%4.8 to 7.5 | not dated | labs.scale.com |
| 23 | claude-sonnet-4-5-20250929_unknownAnthropic | closed | effort unknown | 6.0%4.6 to 7.3 | not dated | labs.scale.com |
| 24 | o1-2024-12-17_unknownOpenAI | closed | effort unknown | 5.7%4.3 to 7.0 | not dated | labs.scale.com |
| 25 | claude-opus-4-20250514_unknownAnthropic | closed | effort unknown | 5.6%4.3 to 6.9 | not dated | labs.scale.com |
| 26 | gemini-2.5-pro-preview-06-05Google DeepMind | closed | default | 5.6%4.3 to 6.9 | not dated | labs.scale.com |
| 27 | claude-3-7-sonnet-20250219_unknownAnthropic | closed | effort unknown | 4.2%3.1 to 5.4 | not dated | labs.scale.com |
| 28 | gemini-2.5-pro-exp-03-25Google DeepMind | closed | default | 4.1%3.0 to 5.3 | not dated | labs.scale.com |
| 29 | moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshot | open | default | 3.4%2.4 to 4.4 | not dated | labs.scale.com |
| 30 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 3.2%2.2 to 4.2 | not dated | labs.scale.com |
| 31 | claude-sonnet-4-20250514_unknownAnthropic | closed | effort unknown | 3.1%2.1 to 4.1 | not dated | labs.scale.com |
| 32 | gemini-3.1-flash-liteGoogle | closed | default | 3.0%2.1 to 4.0 | not dated | labs.scale.com |
| 33 | gemini-2.5-flash-preview-05-20Google DeepMind | closed | default | 2.7%1.8 to 3.6 | not dated | labs.scale.com |
| 34 | gemini-2.5-pro-preview-05-06Google DeepMind | closed | default | 2.4%1.5 to 3.2 | not dated | labs.scale.com |
| 35 | claude-3-7-sonnet-20250219Anthropic | closed | default | 2.3%1.4 to 3.1 | not dated | labs.scale.com |
| 36 | gpt-4.1-2025-04-14OpenAI | closed | default | 2.2%1.3 to 3.0 | not dated | labs.scale.com |
| 37 | gpt-5.1-2025-11-13_noneOpenAI | closed | effort none | 1.9%1.1 to 2.7 | not dated | labs.scale.com |
| 38 | gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Google | closed | default | 1.1%0.5 to 1.7 | not dated | labs.scale.com |
| 39 | claude-3-5-sonnet-20241022Anthropic | closed | default | 0.9%0.4 to 1.5 | not dated | labs.scale.com |
| 40 | pixtral-large-2411Mistral AIno page match | open | default | 0.8%0.3 to 1.4 | not dated | labs.scale.com |
| 41 | claude-3-opus-20240229Anthropic | closed | default | 0.8%0.4 to 1.3 | not dated | labs.scale.com |
| 42 | gpt-4o-2024-11-20OpenAI | closed | default | 0.8%0.4 to 1.2 | not dated | labs.scale.com |
| 43 | gemini-2.0-pro-exp-02-05Google DeepMind | closed | default | 0.7%0.2 to 1.2 | not dated | labs.scale.com |
| 44 | gemini-2.0-flash-02-05Google DeepMind,Google | closed | default | 0.6%0.2 to 1.1 | not dated | labs.scale.com |
| 45 | meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AI | open | default | 0.6%0.1 to 1.0 | not dated | labs.scale.com |
| 46 | meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AI | open | default | 0.4%0.0 to 0.7 | not dated | labs.scale.com |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method