HLE
ReportedHumanity's Last Exam: hard questions written by experts across many academic subjects, many with a single short answer.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Score shown
- Epoch's column "Accuracy", as a percentage, higher is better. Error bars as the source gives them.
- Results
- 56 in all: 4 for open-weight models, 52 for closed ones. 4 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K2.5, 24.4%, number 18 of 56 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | gpt-6-astra_unknownOpenAI | closed | effort unknown | 54.8%±1.9 | not dated | via Epoch AI |
| 2 | claude-fable-5-1_xhighAnthropic | closed | effort xhigh | 46.5%±2.0 | not dated | via Epoch AI |
| 3 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 46.4%±2.0 | not dated | via Epoch AI |
| 4 | claude-sonnet-5-5_xhighAnthropic | closed | effort xhigh | 46.1%±1.9 | not dated | via Epoch AI |
| 5 | gemini-3.8-flash_unknownGoogle DeepMind | closed | effort unknown | 44.5%±2.0 | not dated | via Epoch AI |
| 6 | gpt-5.4-pro-2026-03-05_unknownOpenAI | closed | effort unknown | 44.3%±1.9 | not dated | via Epoch AI |
| 7 | muse-sparkMeta AI | closed | default | 40.6%±1.9 | not dated | via Epoch AI |
| 8 | gemini-3-pro-previewGoogle DeepMind | closed | default | 37.5%±1.9 | not dated | via Epoch AI |
| 9 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 36.2%±1.9 | not dated | via Epoch AI |
| 10 | claude-opus-4-7_unknownAnthropic | closed | effort unknown | 36.2%±1.9 | not dated | via Epoch AI |
| 11 | claude-opus-4-6_maxAnthropic | closed | effort max | 34.4%±1.9 | not dated | via Epoch AI |
| 12 | gpt-5-pro-2025-10-06_unknownOpenAI | closed | effort unknown | 31.6%±1.8 | not dated | via Epoch AI |
| 13 | claude-haiku-5-5_unknownAnthropic | closed | effort unknown | 30.1%±1.8 | not dated | via Epoch AI |
| 14 | gpt-5.2-2025-12-11_unknownOpenAI | closed | effort unknown | 27.8%±1.8 | not dated | via Epoch AI |
| 15 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 25.3%±1.7 | not dated | via Epoch AI |
| 16 | gpt-5-2025-08-07_unknownOpenAI | closed | effort unknown | 25.3%±1.7 | not dated | via Epoch AI |
| 17 | claude-opus-4-5-20251101_unknownAnthropic | closed | effort unknown | 25.2%±1.7 | not dated | via Epoch AI |
| 18 | moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshot | open | default | 24.4%±1.8 | not dated | via Epoch AI |
| 19 | gpt-5.1-2025-11-13_unknownOpenAI | closed | effort unknown | 23.7%±1.7 | not dated | via Epoch AI |
| 20 | gemini-2.5-pro-preview-06-05Google DeepMind | closed | default | 21.6%±1.6 | not dated | via Epoch AI |
| 21 | o3-2025-04-16_highOpenAI | closed | effort high | 20.3%±1.6 | not dated | via Epoch AI |
| 22 | gpt-5-mini-2025-08-07_unknownOpenAI | closed | effort unknown | 19.4%±1.6 | not dated | via Epoch AI |
| 23 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 19.2%±1.5 | not dated | via Epoch AI |
| 24 | claude-opus-4-6Anthropic | closed | default | 19.0%±1.5 | not dated | via Epoch AI |
| 25 | gemini-2.5-pro-exp-03-25Google DeepMind | closed | default | 18.2%±1.5 | not dated | via Epoch AI |
| 26 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 18.1%±1.5 | not dated | via Epoch AI |
| 27 | gemini-2.5-pro-preview-05-06Google DeepMind | closed | default | 17.8%±1.5 | not dated | via Epoch AI |
| 28 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 14.3%±1.4 | not dated | via Epoch AI |
| 29 | claude-opus-4-5-20251101_unknownAnthropic | closed | effort unknown | 14.2%±1.4 | not dated | via Epoch AI |
| 30 | claude-sonnet-4-5-20250929_unknownAnthropic | closed | effort unknown | 13.7%±1.4 | not dated | via Epoch AI |
| 31 | gemini-2.5-flash-preview-04-17Google DeepMind | closed | default | 12.1%±1.3 | not dated | via Epoch AI |
| 32 | claude-opus-4-1-20250805_unknownAnthropic | closed | effort unknown | 11.5%±1.3 | not dated | via Epoch AI |
| 33 | gemini-2.5-flash-preview-05-20Google DeepMind | closed | default | 11.0%±1.2 | not dated | via Epoch AI |
| 34 | claude-opus-4-20250514_unknownAnthropic | closed | effort unknown | 10.7%±1.2 | not dated | via Epoch AI |
| 35 | gemini-3.1-flash-liteGoogle | closed | default | 8.6%±1.1 | not dated | via Epoch AI |
| 36 | zai-org/GLM-4.5Epoch: glm-4.5, Z.ai (Zhipu AI),Tsinghua University | open | default | 8.3%±1.1 | not dated | via Epoch AI |
| 37 | zai-org/GLM-4.5-AirEpoch: GLM-4.5-Air, Z.ai (Zhipu AI),Tsinghua University | open | default | 8.1%±1.1 | not dated | via Epoch AI |
| 38 | o1-pro-2025-03-19OpenAI | closed | default | 8.1%±1.1 | not dated | via Epoch AI |
| 39 | claude-3-7-sonnet-20250219_unknownAnthropic | closed | effort unknown | 8.0%±1.1 | not dated | via Epoch AI |
| 40 | o1-2024-12-17_unknownOpenAI | closed | effort unknown | 8.0%±1.1 | not dated | via Epoch AI |
| 41 | claude-opus-4-1-20250805_unknownAnthropic | closed | effort unknown | 7.9%±1.1 | not dated | via Epoch AI |
| 42 | claude-sonnet-4-20250514_unknownAnthropic | closed | effort unknown | 7.8%±1.1 | not dated | via Epoch AI |
| 43 | claude-sonnet-4-5-20250929_unknownAnthropic | closed | effort unknown | 7.5%±1.0 | not dated | via Epoch AI |
| 44 | gpt-5.1-2025-11-13_noneOpenAI | closed | effort none | 6.8%±1.0 | not dated | via Epoch AI |
| 45 | claude-opus-4-20250514_unknownAnthropic | closed | effort unknown | 6.7%±1.0 | not dated | via Epoch AI |
| 46 | gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Google | closed | default | 6.6%±1.0 | not dated | via Epoch AI |
| 47 | meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AI | open | default | 5.7%±0.9 | not dated | via Epoch AI |
| 48 | claude-sonnet-4-20250514_unknownAnthropic | closed | effort unknown | 5.5%±0.9 | not dated | via Epoch AI |
| 49 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 5.4%±0.9 | not dated | via Epoch AI |
| 50 | gpt-4.1-2025-04-14OpenAI | closed | default | 5.4%±0.9 | not dated | via Epoch AI |
| 51 | gemini-1.5-pro-002Google DeepMind | closed | default | 4.6%±0.8 | not dated | via Epoch AI |
| 52 | mistral-medium-2505Mistral AI | closed | default | 4.5%±0.8 | not dated | via Epoch AI |
| 53 | amazon.nova-pro-v1:0Amazon | closed | default | 4.4%±0.8 | not dated | via Epoch AI |
| 54 | claude-3-5-sonnet-20241022Anthropic | closed | default | 4.1%±0.8 | not dated | via Epoch AI |
| 55 | amazon.nova-lite-v1:0Amazon | closed | default | 3.6%±0.7 | not dated | via Epoch AI |
| 56 | gpt-4o-2024-11-20OpenAI | closed | default | 2.7%±0.6 | not dated | via Epoch AI |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method