Benchmarks
78 eval sets from Epoch AI's benchmark data: 18 that Epoch AI runs itself, with logs, and 60 collected from other leaderboards. Each set is one kind of score, and kinds are never combined. Closed models are kept as reference rows. Method and definitions
Re-run by Epoch AI
Re-run by Epoch AIRun independently by Epoch AI, with public logs. Our own re-runs are not built yet.
| Eval set | Open models | Results | Best open model | Best closed | Latest result |
|---|---|---|---|---|---|
| GPQA diamondMultiple-choice questions in biology, physics and chemistry, written by people with or studying for PhDs and hard to answer by searching the web. Diamond is the subset where expert validators agreed on the answer. | 110 | 341 | kimi-k3.1-pro-together_max 94.2% | gpt-6.2-astra_max 96.8% | 13 Nov 2026 |
| OTIS Mock AIME 2024-2025Mock exams in the style of the AIME, a US mathematics competition, where every answer is a whole number. | 94 | 318 | kimi-k3.1-pro-together_max 100.0% | claude-fable-5_high 100.0% | 13 Nov 2026 |
| MATH level 5The hardest level (5) of the MATH dataset of competition mathematics problems, each with a single final answer. | 43 | 108 | DeepSeek-R1-0528 96.6% | gpt-5-2025-08-07_high 98.1% | 30 Oct 2025 |
| Mystery Game PuzzlesNo description yet. | 22 | 148 | deepseek-v4.1-flash-1017_max 49.0% | gpt-6.2-astra_max 95.0% | 13 Nov 2026 |
| SimpleQA VerifiedNo description yet. | 21 | 105 | kimi-k3.1-pro-together_max 62.2% | claude-fable-5-2_max 94.0% | 13 Nov 2026 |
| FrontierMath-Tiers-1-3-v2-PrivateTiers 1 to 3 of FrontierMath (second version): unpublished mathematics problems written for Epoch AI, from advanced undergraduate to research level. | 20 | 127 | kimi-k3.1-pro-together_max 78.2% | claude-fable-5-2_max 95.1% | 13 Nov 2026 |
| Chess PuzzlesNo description yet. | 18 | 166 | deepseek-v4.1-flash_max 48.0% | claude-fable-5-2_max 79.0% | 13 Nov 2026 |
| FrontierMath-Tier-4-v2-PrivateThe hardest tier of FrontierMath (second version): unpublished research-level mathematics problems written for Epoch AI. | 16 | 83 | kimi-k3.1-pro-together_max 65.9% | claude-fable-5-2_max 100.0% | 13 Nov 2026 |
| FrontierMath-2025-02-28-PrivateUnpublished research-level mathematics problems written by mathematicians for Epoch AI, with answers that can be checked automatically. | 13 | 101 | kimi-k2.6 39.0% | gpt-5.5-pro-pre-release_high 52.4% | 8 Jun 2026 |
| FrontierMath-Tier-4-2025-07-01-PrivateThe hardest tier of FrontierMath: unpublished research-level mathematics problems written for Epoch AI. | 9 | 72 | kimi-k2.6 14.6% | gdm-ai-co-mathematician 47.9% | 8 Jun 2026 |
| Furniture AssemblyNo description yet. | 6 | 41 | kimi-k3.1-pro-together_max 55.8% | claude-fable-5-2_max 90.0% | 13 Nov 2026 |
| SWE-Bench verifiedReal issues from open-source Python repositories: the model must write a code change that makes the repository's tests pass. Verified is a subset checked by people to be solvable as stated. | 6 | 35 | glm-5.2_max 78.7% | claude-opus-4-7_max 83.5% | 25 Jun 2026 |
| MirrorCodeNo description yet. | 5 | 47 | deepseek-v4.1-flash_max 64.4% | claude-sonnet-5-5_max 97.8% | 2 Nov 2026 |
| EBR-benchNo description yet. | 3 | 33 | mistral-large-4-0-oss_high 18.1% | gpt-6.2-astra_max 92.4% | 13 Nov 2026 |
| FrontierMath-ErdosNo description yet. | 1 | 16 | kimi-k3_max 0.0% | gpt-6.2-astra_max 10.3% | 7 Nov 2026 |
| SpireClimbNo description yet. | 0 | 11 | none | gpt-6.2-astra_max 41.3% | 13 Nov 2026 |
| SpireClimb (Slay the Spire 1, Ascension 20 Heart)No description yet. | 0 | 11 | none | gpt-6-astra_max 33.3% | 13 Nov 2026 |
| SpireClimb (Slay the Spire 2, Ascension 10)No description yet. | 0 | 11 | none | gpt-6.2-astra_max 49.3% | 13 Nov 2026 |
Reported by other leaderboards
ReportedPublished by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
| Eval set | Open models | Results | Best open model | Best closed | Latest result |
|---|---|---|---|---|---|
| MMLUMultiple-choice questions across 57 subjects, from school level to professional level. | 88 | 217 | DeepSeek-V3 87.2% | gpt-4o-2024-11-20 88.1% | not dated |
| CritPtNo description yet. | 87 | 271 | mimo-v2.6-pro 26.6% | gemini-4-carbon_high 36.0% | not dated |
| DTBenchNo description yet. | 75 | 241 | deepseek-v4-pro-0813_unknown 93.9% | claude-fable-5-2_unknown 99.2% | not dated |
| GSM8KGrade-school mathematics word problems that need a few steps of arithmetic. | 70 | 170 | DeepSeek-Coder-V2-Instruct 94.5% | gpt-4-0314 92.0% | not dated |
| ARC AI2Multiple-choice science questions from school exams (the AI2 Reasoning Challenge); this is the harder Challenge set. | 61 | 134 | DeepSeek-V3 95.3% | gpt-3.5-turbo-1106 87.4% | not dated |
| LMCANo description yet. | 59 | 203 | kimi-k3.1-pro_max 57.6% | claude-fable-5-2_xhigh 70.6% | not dated |
| WinograndeFill-in-the-blank sentences where the right answer needs common sense about who or what a word refers to. | 56 | 137 | Llama-3.1-405B 89.2% | claude-3-opus-20240229 88.5% | not dated |
| HellaSwagChoosing the most sensible way to continue a short description of an everyday situation. | 52 | 113 | Llama-3.1-405B 89.2% | gpt-4-0314 95.3% | not dated |
| BoolQYes or no questions about a short passage. | 48 | 136 | T5-11B 91.2% | Inflection-1 89.7% | not dated |
| WeirdMLNo description yet. | 48 | 173 | kimi-k3_max 82.6% | gpt-6-astra_promax 93.6% | not dated |
| PIQAPhysical common sense: choosing the better of two ways to do an everyday task. | 46 | 113 | Phi-3.5-MoE-instruct 88.6% | gpt-4o-mini-2024-07-18 88.7% | not dated |
| BBHBIG-Bench Hard: a set of the hardest tasks from the BIG-Bench collection, covering logic, arithmetic and language puzzles. | 42 | 88 | DeepSeek-V3 87.5% | gemini-1.5-pro-001 89.2% | not dated |
| SimpleBenchNo description yet. | 33 | 125 | deepseek-v4.1-flash_unknown 66.7% | claude-fable-5-2_unknown 94.9% | not dated |
| OpenBookQAElementary science questions that need a given fact plus common knowledge to answer. | 31 | 70 | Phi-3-mini-4k-instruct 88.0% | gpt-3.5-turbo-1106 86.0% | not dated |
| ProofBenchNo description yet. | 30 | 85 | kimi-k3.1-pro_unknown 91.0% | claude-fable-5-1_max 100.0% | not dated |
| ARC-AGIVisual puzzles on small coloured grids: from a few examples the model must find the rule and apply it to a new grid. | 26 | 318 | inkling-2-small_xhigh 96.5% | claude-fable-5_max 98.5% | not dated |
| ARC-AGI-2The second, harder set of ARC-AGI grid puzzles: from a few examples the model must find the rule and apply it to a new grid. | 26 | 299 | inkling-2-small_xhigh 81.7% | gpt-6-astra_max 95.0% | not dated |
| Fiction.LiveBenchNo description yet. | 24 | 61 | kimi-k2.5 86.1% | gpt-5-2025-08-07_medium 97.2% | not dated |
| TriviaQATrivia questions, answered without being given the source text. | 24 | 115 | Llama-2-70b-hf 87.6% | claude-2.0 87.5% | not dated |
| APEX-AgentsNo description yet. | 22 | 66 | deepseek-v4.1-flash-1017_max 69.8% | gemini-4-carbon_unknown 84.2% | not dated |
| LiveBenchQuestions in mathematics, coding, reasoning, data analysis, language and instruction following, refreshed regularly so that they are less likely to be in training data. | 21 | 54 | QwQ-32B 72.0% | gemini-2.5-pro-exp-03-25 82.3% | not dated |
| Aider polyglotCoding exercises in several programming languages, where the model must edit code so that the exercise's tests pass. | 20 | 72 | deepseek-reasoner 74.2% | gpt-5-2025-08-07_high 88.0% | 3 Oct 2025 |
| LAMBADAPredicting the last word of a passage, where the word can only be guessed from the wider context. | 20 | 53 | falcon-180B 79.8% | Megatron-Turing NLG 530B 87.2% | not dated |
| BalrogNo description yet. | 16 | 41 | DeepSeek-R1 34.9% | gpt-6-astra_max 68.3% | 20 Sep 2026 |
| Terminal BenchTasks an agent must complete in a command-line terminal, checked by tests. | 16 | 202 | glm-5 52.4% | gpt-5.5_unknown 84.7% | 14 May 2026 |
| Surface Evolver BenchNo description yet. | 15 | 29 | kimi-k3_unknown 95.0% | claude-fable-5_high 95.0% | not dated |
| FrontierCodeNo description yet. | 14 | 46 | kimi-k3_unknown 44.2% | claude-opus-5-5_medium 54.6% | not dated |
| Lech Mazur WritingNo description yet. | 14 | 49 | Kimi-K2-Instruct 85.6% | gpt-5-2025-08-07_medium 86.0% | not dated |
| ScienceQAMultiple-choice science questions from school curricula, many with an image. | 10 | 26 | Phi-3.5-vision-instruct 91.3% | gpt-4o-2024-05-13 88.5% | not dated |
| VideoMMENo description yet. | 10 | 50 | Qwen2.5-VL-72B-Instruct 73.5% | gemini-1.5-pro-001 75.0% | not dated |
| METR Time HorizonsNo description yet. | 9 | 49 | kimi-k2-thinking 59.2% | claude-mythos-preview-early 85.2% | not dated |
| ANLINatural language inference (does one sentence follow from another) with examples collected to fool earlier models. | 8 | 15 | Phi-3-small-8k-instruct 58.1% | gpt-3.5-turbo-1106 58.1% | not dated |
| CL-benchNo description yet. | 7 | 23 | kimi-k2.5 19.3% | gpt-5.4-2026-03-05_xhigh 27.9% | not dated |
| GBAEvalNo description yet. | 7 | 28 | kimi-k3_unknown 48.3% | claude-opus-5-5_unknown 90.6% | 14 Nov 2026 |
| CL-bench LifeNo description yet. | 6 | 17 | deepseek-v4-pro_high 13.5% | gpt-5.5_high 22.2% | not dated |
| GDP.pdfNo description yet. | 6 | 19 | kimi-k3.1-pro_max 46.6% | gemini-4-carbon_high 56.4% | not dated |
| GeoBenchNo description yet. | 6 | 51 | Qwen2.5-VL-72B-Instruct 62.0% | claude-fable-5_high 89.0% | not dated |
| WeirdML v3No description yet. | 6 | 27 | deepseek-v4.1-flash-1017_high 19.7% | claude-fable-5-2_xhigh 65.3% | not dated |
| DeepSWENo description yet. | 5 | 69 | glm-5.3_max 69.0% | gpt-6-astra_xhigh 74.1% | not dated |
| FrontierSWENo description yet. | 5 | 25 | glm-5.3_max 30.2% | claude-fable-5-2_max 69.9% | not dated |
| The Agent CompanyNo description yet. | 5 | 16 | DeepSeek-V3.2-Exp 42.9% | gemini-2.5-flash-preview-09-2025 41.1% | 13 Oct 2025 |
| EnigmaEvalNo description yet. | 4 | 46 | kimi-k2.5 3.4% | claude-fable-5_high 39.3% | not dated |
| HLEHumanity's Last Exam: hard questions written by experts across many academic subjects, many with a single short answer. | 4 | 56 | kimi-k2.5 24.4% | gpt-6-astra_unknown 54.8% | not dated |
| PostTrainBenchNo description yet. | 4 | 22 | glm-5.3-flash_max 36.8% | claude-fable-5-1_max 44.6% | not dated |
| CSQA2No description yet. | 3 | 6 | T5-11B 67.8% | gpt-3.5-turbo-0613 57.0% | not dated |
| CursorBenchNo description yet. | 3 | 94 | deepseek-v4.1-flash-1017_max 47.7% | gemini-4-carbon_high 60.4% | not dated |
| CybenchCapture-the-flag security tasks taken from professional competitions. | 3 | 22 | Llama-3.1-405B-Instruct 7.5% | claude-opus-4-6_unknown 93.0% | not dated |
| ExploitBenchNo description yet. | 3 | 20 | kimi-k2.6 18.4% | claude-mythos-preview-early 73.8% | not dated |
| GSO-BenchNo description yet. | 3 | 42 | Kimi-K2-Instruct 4.9% | claude-fable-5-1_unknown 88.2% | 27 Sep 2026 |
| SpatialViz-BenchNo description yet. | 3 | 8 | Llama-4-Scout-17B-16E-Instruct 34.2% | gemini-2.5-pro 44.7% | not dated |
| Blueprint-Bench 2No description yet. | 2 | 38 | kimi-k3_unknown 29.5% | gemini-4-carbon_unknown 58.6% | not dated |
| OSWorldTasks done in a real desktop operating system through the screen, keyboard and mouse. | 2 | 20 | kimi-k2.5 63.3% | claude-sonnet-4-6 72.1% | 3 Feb 2026 |
| OSWorld 2.0No description yet. | 2 | 16 | kimi-k2.6 4.6% | claude-opus-5_max 31.4% | not dated |
| SuperGLUEA suite of English language understanding tasks such as question answering and inference. | 2 | 10 | T5-11B 88.9% | text-davinci-001 71.8% | not dated |
| DeepResearch BenchNo description yet. | 1 | 41 | DeepSeek-R1-0528 35.1% | claude-opus-4-6_high 55.3% | not dated |
| MindCubeNo description yet. | 1 | 5 | gemma-3-12b-it 46.7% | claude-sonnet-4-20250514_unknown 44.8% | not dated |
| Remote Labor IndexNo description yet. | 1 | 20 | deepseek-v4.1-flash_unknown 9.6% | claude-fable-5-2_unknown 41.3% | not dated |
| CadEvalNo description yet. | 0 | 15 | none | o3-2025-04-16_medium 74.0% | not dated |
| GDPvalWork tasks from many occupations, where the model's output is compared with an expert's by expert graders. | 0 | 11 | none | gpt-5.2-2025-12-11_none 49.7% | not dated |
| VPCTNo description yet. | 0 | 38 | none | gemini-3-pro-preview 91.0% | not dated |
Data from Epoch AI, used under CC-BY 4.0, fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI's benchmarking hub.
Rows from Artificial Analysis and LMArena are in Epoch's data but not copied here; see Artificial Analysis and the Arena leaderboard directly. Files whose score is not a fraction where higher is better are left out; the list is in our curation files.