METR Time Horizons
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
- Score shown
- Epoch's column "average_score", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
- Results
- 49 in all: 9 for open-weight models, 40 for closed ones. 8 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K2-Thinking, 59.2%, number 24 of 49 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-mythos-preview-earlyAnthropic | closed | default | 85.2% | not dated | metr.org |
| 2 | claude-opus-4-6_unknownAnthropic | closed | effort unknown | 78.9% | not dated | via Epoch AI |
| 3 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 77.0% | not dated | metr.org |
| 4 | gpt-5.2-2025-12-11_highOpenAI | closed | effort high | 75.3% | not dated | via Epoch AI |
| 5 | claude-opus-4-5-20251101_16KAnthropic | closed | default | 75.0% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 6 | gpt-5.3-codexOpenAI | closed | default | 74.5% | not dated | via Epoch AI |
| 7 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 74.3% | not dated | metr.org |
| 8 | claude-opus-4-5-20251101Anthropic | closed | default | 73.0% | not dated | via Epoch AI |
| 9 | gemini-3-pro-previewGoogle DeepMind | closed | default | 71.0% | not dated | via Epoch AI |
| 10 | gpt-5.1-codex-maxOpenAI | closed | default | 70.8% | not dated | metr.org |
| 11 | gpt-5-2025-08-07_mediumOpenAI | closed | effort medium | 69.6% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 12 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 69.4% | not dated | via Epoch AI |
| 13 | claude-sonnet-4-5-20250929_16KAnthropic | closed | default | 67.4% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 14 | claude-opus-4-1-20250805_16KAnthropic | closed | default | 66.8% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 15 | grok-4-0709xAI | closed | default | 66.6% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 16 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 65.4% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 17 | claude-opus-4-20250514_16KAnthropic | closed | default | 63.9% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 18 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 63.9% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 19 | o3-2025-04-16_unknownOpenAI | closed | effort unknown | 63.6% | not dated | via Epoch AI |
| 20 | claude-sonnet-4-20250514_16KAnthropic | closed | default | 62.0% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 21 | claude-opus-4-1-20250805Anthropic | closed | default | 61.6% | not dated | via Epoch AI |
| 22 | claude-opus-4-20250514_unknownAnthropic | closed | effort unknown | 61.5% | not dated | via Epoch AI |
| 23 | claude-3-7-sonnet-20250219_16KAnthropic | closed | default | 60.0% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 24 | moonshotai/Kimi-K2-ThinkingEpoch: kimi-k2-thinking, Moonshot | open | default | 59.2% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 25 | openai/gpt-oss-120bEpoch: gpt-oss-120b, OpenAI | open | default | 56.6% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 26 | claude-3-7-sonnet-20250219Anthropic | closed | default | 55.8% | not dated | via Epoch AI |
| 27 | gemini-2.5-pro-preview-06-05Google DeepMind | closed | default | 55.4% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 28 | deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeek | open | default | 53.8% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 29 | deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeek | open | default | 51.9% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 30 | o1-2024-12-17_mediumOpenAI | closed | effort medium | 51.1% | not dated | metr.org |
| 31 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 49.6% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 32 | deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeek | open | default | 47.4% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 33 | claude-3-5-sonnet-20241022Anthropic | closed | default | 45.2% | not dated | metr.org |
| 34 | o1-preview-2024-09-12OpenAI | closed | default | 45.1% | not dated | metr.org |
| 35 | gpt-4o-2024-05-13OpenAI | closed | default | 40.8% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 36 | claude-3-5-sonnet-20240620Anthropic | closed | default | 40.2% | not dated | metr.org |
| 37 | gpt-4-turbo-2024-04-09OpenAI | closed | default | 36.7% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 38 | gpt-4-0314OpenAI | closed | default | 36.1% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 39 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 35.8% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 40 | gpt-4-0125-previewOpenAI | closed | default | 35.2% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 41 | gpt-4o-2024-05-13OpenAI | closed | default | 33.8% | not dated | via Epoch AI |
| 42 | Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, Alibaba | open | default | 29.9% | not dated | METR - Measuring AI Ability to Complete Long Tasks |
| 43 | claude-3-opus-20240229Anthropic | closed | default | 29.5% | not dated | metr.org |
| 44 | gpt-4-0314OpenAI | closed | default | 29.3% | not dated | via Epoch AI |
| 45 | gpt-4-1106-previewOpenAI | closed | default | 28.9% | not dated | metr.org |
| 46 | gpt-4-turbo-2024-04-09OpenAI | closed | default | 27.2% | not dated | via Epoch AI |
| 47 | gpt-3.5-turbo-instructOpenAI | closed | default | 21.5% | not dated | metr.org |
| 48 | davinci-002OpenAI | closed | default | 16.2% | not dated | metr.org |
| 49 | gpt2-xlOpenAIno page match | open | default | 10.1% | not dated | metr.org |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method