Skip to content
Weights Not Walls
Menu

Benchmarks

78 eval sets from Epoch AI's benchmark data: 18 that Epoch AI runs itself, with logs, and 60 collected from other leaderboards. Each set is one kind of score, and kinds are never combined. Closed models are kept as reference rows. Method and definitions

Re-run by Epoch AI

Re-run by Epoch AI

Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.

Re-run by Epoch AI
Eval setOpen modelsResultsBest open modelBest closedLatest result
GPQA diamondMultiple-choice questions in biology, physics and chemistry, written by people with or studying for PhDs and hard to answer by searching the web. Diamond is the subset where expert validators agreed on the answer.110341kimi-k3.1-pro-together_max 94.2%gpt-6.2-astra_max 96.8%13 Nov 2026
OTIS Mock AIME 2024-2025Mock exams in the style of the AIME, a US mathematics competition, where every answer is a whole number.94318kimi-k3.1-pro-together_max 100.0%claude-fable-5_high 100.0%13 Nov 2026
MATH level 5The hardest level (5) of the MATH dataset of competition mathematics problems, each with a single final answer.43108DeepSeek-R1-0528 96.6%gpt-5-2025-08-07_high 98.1%30 Oct 2025
Mystery Game PuzzlesNo description yet.22148deepseek-v4.1-flash-1017_max 49.0%gpt-6.2-astra_max 95.0%13 Nov 2026
SimpleQA VerifiedNo description yet.21105kimi-k3.1-pro-together_max 62.2%claude-fable-5-2_max 94.0%13 Nov 2026
FrontierMath-Tiers-1-3-v2-PrivateTiers 1 to 3 of FrontierMath (second version): unpublished mathematics problems written for Epoch AI, from advanced undergraduate to research level.20127kimi-k3.1-pro-together_max 78.2%claude-fable-5-2_max 95.1%13 Nov 2026
Chess PuzzlesNo description yet.18166deepseek-v4.1-flash_max 48.0%claude-fable-5-2_max 79.0%13 Nov 2026
FrontierMath-Tier-4-v2-PrivateThe hardest tier of FrontierMath (second version): unpublished research-level mathematics problems written for Epoch AI.1683kimi-k3.1-pro-together_max 65.9%claude-fable-5-2_max 100.0%13 Nov 2026
FrontierMath-2025-02-28-PrivateUnpublished research-level mathematics problems written by mathematicians for Epoch AI, with answers that can be checked automatically.13101kimi-k2.6 39.0%gpt-5.5-pro-pre-release_high 52.4%8 Jun 2026
FrontierMath-Tier-4-2025-07-01-PrivateThe hardest tier of FrontierMath: unpublished research-level mathematics problems written for Epoch AI.972kimi-k2.6 14.6%gdm-ai-co-mathematician 47.9%8 Jun 2026
Furniture AssemblyNo description yet.641kimi-k3.1-pro-together_max 55.8%claude-fable-5-2_max 90.0%13 Nov 2026
SWE-Bench verifiedReal issues from open-source Python repositories: the model must write a code change that makes the repository's tests pass. Verified is a subset checked by people to be solvable as stated.635glm-5.2_max 78.7%claude-opus-4-7_max 83.5%25 Jun 2026
MirrorCodeNo description yet.547deepseek-v4.1-flash_max 64.4%claude-sonnet-5-5_max 97.8%2 Nov 2026
EBR-benchNo description yet.333mistral-large-4-0-oss_high 18.1%gpt-6.2-astra_max 92.4%13 Nov 2026
FrontierMath-ErdosNo description yet.116kimi-k3_max 0.0%gpt-6.2-astra_max 10.3%7 Nov 2026
SpireClimbNo description yet.011nonegpt-6.2-astra_max 41.3%13 Nov 2026
SpireClimb (Slay the Spire 1, Ascension 20 Heart)No description yet.011nonegpt-6-astra_max 33.3%13 Nov 2026
SpireClimb (Slay the Spire 2, Ascension 10)No description yet.011nonegpt-6.2-astra_max 49.3%13 Nov 2026

Reported by other leaderboards

Reported

Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.

Reported by other leaderboards
Eval setOpen modelsResultsBest open modelBest closedLatest result
MMLUMultiple-choice questions across 57 subjects, from school level to professional level.88217DeepSeek-V3 87.2%gpt-4o-2024-11-20 88.1%not dated
CritPtNo description yet.87271mimo-v2.6-pro 26.6%gemini-4-carbon_high 36.0%not dated
DTBenchNo description yet.75241deepseek-v4-pro-0813_unknown 93.9%claude-fable-5-2_unknown 99.2%not dated
GSM8KGrade-school mathematics word problems that need a few steps of arithmetic.70170DeepSeek-Coder-V2-Instruct 94.5%gpt-4-0314 92.0%not dated
ARC AI2Multiple-choice science questions from school exams (the AI2 Reasoning Challenge); this is the harder Challenge set.61134DeepSeek-V3 95.3%gpt-3.5-turbo-1106 87.4%not dated
LMCANo description yet.59203kimi-k3.1-pro_max 57.6%claude-fable-5-2_xhigh 70.6%not dated
WinograndeFill-in-the-blank sentences where the right answer needs common sense about who or what a word refers to.56137Llama-3.1-405B 89.2%claude-3-opus-20240229 88.5%not dated
HellaSwagChoosing the most sensible way to continue a short description of an everyday situation.52113Llama-3.1-405B 89.2%gpt-4-0314 95.3%not dated
BoolQYes or no questions about a short passage.48136T5-11B 91.2%Inflection-1 89.7%not dated
WeirdMLNo description yet.48173kimi-k3_max 82.6%gpt-6-astra_promax 93.6%not dated
PIQAPhysical common sense: choosing the better of two ways to do an everyday task.46113Phi-3.5-MoE-instruct 88.6%gpt-4o-mini-2024-07-18 88.7%not dated
BBHBIG-Bench Hard: a set of the hardest tasks from the BIG-Bench collection, covering logic, arithmetic and language puzzles.4288DeepSeek-V3 87.5%gemini-1.5-pro-001 89.2%not dated
SimpleBenchNo description yet.33125deepseek-v4.1-flash_unknown 66.7%claude-fable-5-2_unknown 94.9%not dated
OpenBookQAElementary science questions that need a given fact plus common knowledge to answer.3170Phi-3-mini-4k-instruct 88.0%gpt-3.5-turbo-1106 86.0%not dated
ProofBenchNo description yet.3085kimi-k3.1-pro_unknown 91.0%claude-fable-5-1_max 100.0%not dated
ARC-AGIVisual puzzles on small coloured grids: from a few examples the model must find the rule and apply it to a new grid.26318inkling-2-small_xhigh 96.5%claude-fable-5_max 98.5%not dated
ARC-AGI-2The second, harder set of ARC-AGI grid puzzles: from a few examples the model must find the rule and apply it to a new grid.26299inkling-2-small_xhigh 81.7%gpt-6-astra_max 95.0%not dated
Fiction.LiveBenchNo description yet.2461kimi-k2.5 86.1%gpt-5-2025-08-07_medium 97.2%not dated
TriviaQATrivia questions, answered without being given the source text.24115Llama-2-70b-hf 87.6%claude-2.0 87.5%not dated
APEX-AgentsNo description yet.2266deepseek-v4.1-flash-1017_max 69.8%gemini-4-carbon_unknown 84.2%not dated
LiveBenchQuestions in mathematics, coding, reasoning, data analysis, language and instruction following, refreshed regularly so that they are less likely to be in training data.2154QwQ-32B 72.0%gemini-2.5-pro-exp-03-25 82.3%not dated
Aider polyglotCoding exercises in several programming languages, where the model must edit code so that the exercise's tests pass.2072deepseek-reasoner 74.2%gpt-5-2025-08-07_high 88.0%3 Oct 2025
LAMBADAPredicting the last word of a passage, where the word can only be guessed from the wider context.2053falcon-180B 79.8%Megatron-Turing NLG 530B 87.2%not dated
BalrogNo description yet.1641DeepSeek-R1 34.9%gpt-6-astra_max 68.3%20 Sep 2026
Terminal BenchTasks an agent must complete in a command-line terminal, checked by tests.16202glm-5 52.4%gpt-5.5_unknown 84.7%14 May 2026
Surface Evolver BenchNo description yet.1529kimi-k3_unknown 95.0%claude-fable-5_high 95.0%not dated
FrontierCodeNo description yet.1446kimi-k3_unknown 44.2%claude-opus-5-5_medium 54.6%not dated
Lech Mazur WritingNo description yet.1449Kimi-K2-Instruct 85.6%gpt-5-2025-08-07_medium 86.0%not dated
ScienceQAMultiple-choice science questions from school curricula, many with an image.1026Phi-3.5-vision-instruct 91.3%gpt-4o-2024-05-13 88.5%not dated
VideoMMENo description yet.1050Qwen2.5-VL-72B-Instruct 73.5%gemini-1.5-pro-001 75.0%not dated
METR Time HorizonsNo description yet.949kimi-k2-thinking 59.2%claude-mythos-preview-early 85.2%not dated
ANLINatural language inference (does one sentence follow from another) with examples collected to fool earlier models.815Phi-3-small-8k-instruct 58.1%gpt-3.5-turbo-1106 58.1%not dated
CL-benchNo description yet.723kimi-k2.5 19.3%gpt-5.4-2026-03-05_xhigh 27.9%not dated
GBAEvalNo description yet.728kimi-k3_unknown 48.3%claude-opus-5-5_unknown 90.6%14 Nov 2026
CL-bench LifeNo description yet.617deepseek-v4-pro_high 13.5%gpt-5.5_high 22.2%not dated
GDP.pdfNo description yet.619kimi-k3.1-pro_max 46.6%gemini-4-carbon_high 56.4%not dated
GeoBenchNo description yet.651Qwen2.5-VL-72B-Instruct 62.0%claude-fable-5_high 89.0%not dated
WeirdML v3No description yet.627deepseek-v4.1-flash-1017_high 19.7%claude-fable-5-2_xhigh 65.3%not dated
DeepSWENo description yet.569glm-5.3_max 69.0%gpt-6-astra_xhigh 74.1%not dated
FrontierSWENo description yet.525glm-5.3_max 30.2%claude-fable-5-2_max 69.9%not dated
The Agent CompanyNo description yet.516DeepSeek-V3.2-Exp 42.9%gemini-2.5-flash-preview-09-2025 41.1%13 Oct 2025
EnigmaEvalNo description yet.446kimi-k2.5 3.4%claude-fable-5_high 39.3%not dated
HLEHumanity's Last Exam: hard questions written by experts across many academic subjects, many with a single short answer.456kimi-k2.5 24.4%gpt-6-astra_unknown 54.8%not dated
PostTrainBenchNo description yet.422glm-5.3-flash_max 36.8%claude-fable-5-1_max 44.6%not dated
CSQA2No description yet.36T5-11B 67.8%gpt-3.5-turbo-0613 57.0%not dated
CursorBenchNo description yet.394deepseek-v4.1-flash-1017_max 47.7%gemini-4-carbon_high 60.4%not dated
CybenchCapture-the-flag security tasks taken from professional competitions.322Llama-3.1-405B-Instruct 7.5%claude-opus-4-6_unknown 93.0%not dated
ExploitBenchNo description yet.320kimi-k2.6 18.4%claude-mythos-preview-early 73.8%not dated
GSO-BenchNo description yet.342Kimi-K2-Instruct 4.9%claude-fable-5-1_unknown 88.2%27 Sep 2026
SpatialViz-BenchNo description yet.38Llama-4-Scout-17B-16E-Instruct 34.2%gemini-2.5-pro 44.7%not dated
Blueprint-Bench 2No description yet.238kimi-k3_unknown 29.5%gemini-4-carbon_unknown 58.6%not dated
OSWorldTasks done in a real desktop operating system through the screen, keyboard and mouse.220kimi-k2.5 63.3%claude-sonnet-4-6 72.1%3 Feb 2026
OSWorld 2.0No description yet.216kimi-k2.6 4.6%claude-opus-5_max 31.4%not dated
SuperGLUEA suite of English language understanding tasks such as question answering and inference.210T5-11B 88.9%text-davinci-001 71.8%not dated
DeepResearch BenchNo description yet.141DeepSeek-R1-0528 35.1%claude-opus-4-6_high 55.3%not dated
MindCubeNo description yet.15gemma-3-12b-it 46.7%claude-sonnet-4-20250514_unknown 44.8%not dated
Remote Labor IndexNo description yet.120deepseek-v4.1-flash_unknown 9.6%claude-fable-5-2_unknown 41.3%not dated
CadEvalNo description yet.015noneo3-2025-04-16_medium 74.0%not dated
GDPvalWork tasks from many occupations, where the model's output is compared with an expert's by expert graders.011nonegpt-5.2-2025-12-11_none 49.7%not dated
VPCTNo description yet.038nonegemini-3-pro-preview 91.0%not dated

Data from Epoch AI, used under CC-BY 4.0, fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI's benchmarking hub.

Rows from Artificial Analysis and LMArena are in Epoch's data but not copied here; see Artificial Analysis and the Arena leaderboard directly. Files whose score is not a fraction where higher is better are left out; the list is in our curation files.