GPQA diamond
Re-run by Epoch AIMultiple-choice questions in biology, physics and chemistry, written by people with or studying for PhDs and hard to answer by searching the web. Diamond is the subset where expert validators agreed on the answer.
- Kind
- Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
- Score shown
- Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 25.0%. Error bars as the source gives them.
- Results
- 341 in all: 131 for open-weight models, 206 for closed ones, 4 where Epoch does not say. 109 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K3.1-Pro, 94.2%, number 15 of 341 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | gpt-6.2-astra_maxOpenAI | closed | effort max | 96.8%±1.2 | 27 Oct 2026 | Epoch AI |
| 2 | gemini-4-hydrogen_xhighGoogle DeepMind | closed | effort xhigh | 96.8%±1.2 | 13 Nov 2026 | Epoch AI log |
| 3 | gemini-4-carbon_highGoogle DeepMind | closed | effort high | 96.5%±1.2 | 2 Nov 2026 | Epoch AI |
| 4 | gemini-4-argon_highGoogle DeepMind | closed | effort high | 96.4%±1.2 | 13 Oct 2026 | Epoch AI log |
| 5 | claude-fable-5-2-fallback_maxAnthropic | closed | effort max | 96.0%±1.3 | 6 Nov 2026 | Epoch AI |
| 6 | gpt-6-astra_maxOpenAI | closed | effort max | 95.8%±1.4 | 30 Aug 2026 | Epoch AI log |
| 7 | claude-sonnet-5-5_maxAnthropic | closed | effort max | 95.6%±1.4 | 29 Sep 2026 | Epoch AI log |
| 8 | gemini-3.8-flash_highGoogle DeepMind | closed | effort high | 95.4%±1.4 | 2 Sep 2026 | Epoch AI log |
| 9 | gpt-6.1-sol_maxOpenAI | closed | effort max | 95.4%±1.4 | 29 Sep 2026 | Epoch AI log |
| 10 | gemini-3.7-flash_highGoogle DeepMind | closed | effort high | 94.8%±1.3 | 14 Aug 2026 | Epoch AI log |
| 11 | gpt-5.4-pro-2026-03-05_xhighOpenAI | closed | effort xhigh | 94.6%±1.6 | 20 Mar 2026 | Epoch AI |
| 12 | gemini-3.1-pro-preview_highGoogle DeepMind | closed | effort high | 94.4%±1.6 | 6 Aug 2026 | Epoch AI log |
| 13 | gpt-6-sol_maxOpenAI | closed | effort max | 94.3%±1.5 | 22 Sep 2026 | Epoch AI log |
| 14 | gpt-6.1-terra_maxOpenAI | closed | effort max | 94.3%±1.5 | 26 Oct 2026 | Epoch AI log |
| 15 | moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro-together_max, Moonshot | open | effort max | 94.2%±1.4 | 6 Nov 2026 | Epoch AI log |
| 16 | gemini-3.6-flash_highGoogle DeepMind | closed | effort high | 94.1%±1.4 | 2 Aug 2026 | Epoch AI log |
| 17 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 94.1%±1.7 | 20 Feb 2026 | Epoch AI log |
| 18 | claude-opus-5-5-fallback_maxAnthropic | closed | effort max | 94.1%±1.5 | 22 Sep 2026 | Epoch AI |
| 19 | gpt-5.5-pre-release_xhighOpenAI | closed | effort xhigh | 94.0%±1.5 | 24 Apr 2026 | Epoch AI log |
| 20 | grok-4.6_highxAI | closed | effort high | 94.0%±1.4 | 12 Aug 2026 | Epoch AI log |
| 21 | gpt-5.5-pro-pre-release_xhighOpenAI | closed | effort xhigh | 93.9%±1.6 | 24 Apr 2026 | Epoch AI log |
| 22 | claude-fable-5-2_maxAnthropic | closed | effort max | 93.9%±1.6 | 6 Nov 2026 | Epoch AI |
| 23 | claude-fable-5-fallback_maxAnthropic | closed | effort max | 93.9%±1.7 | 6 Aug 2026 | Epoch AI |
| 24 | claude-opus-5_maxAnthropic | closed | effort max | 93.9%±1.5 | 24 Jul 2026 | Epoch AI log |
| 25 | gpt-5.6-sol_maxOpenAI | closed | effort max | 93.5%±1.6 | 9 Jul 2026 | Epoch AI log |
| 26 | grok-4.5_highxAI | closed | effort high | 93.4%±1.4 | 8 Jul 2026 | Epoch AI log |
| 27 | gpt-5.6-terra_maxOpenAI | closed | effort max | 93.3%±1.5 | 9 Jul 2026 | Epoch AI log |
| 28 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 93.3%±1.8 | 6 Mar 2026 | Epoch AI log |
| 29 | deepseek-ai/DeepSeek-V4.1-Flash-1017Epoch: deepseek-v4.1-flash-1017_max, DeepSeek | open | effort max | 93.2%±1.6 | 19 Oct 2026 | Epoch AI log |
| 30 | grok-4.6_xhighxAI | closed | effort xhigh | 93.2%±1.5 | 14 Aug 2026 | Epoch AI log |
| 31 | doubao-seed-evolving_highByteDance | closed | effort high | 93.1%±1.6 | 2 Nov 2026 | Epoch AI log |
| 32 | moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshot | open | effort max | 93.1%±1.5 | 16 Jul 2026 | Epoch AI log |
| 33 | claude-opus-5Anthropic | closed | default | 92.9%±1.8 | 6 Aug 2026 | Epoch AI log |
| 34 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 92.8%±1.6 | 22 May 2026 | Epoch AI log |
| 35 | grok-4.7_xhighxAI | closed | effort xhigh | 92.7%±1.6 | 22 Sep 2026 | Epoch AI log |
| 36 | qwen3.8-max_xhighAlibaba | closed | effort xhigh | 92.7%±1.7 | 4 Aug 2026 | Epoch AI log |
| 37 | gemini-3-pro-previewGoogle DeepMind | closed | default | 92.6%±1.7 | 19 Nov 2025 | Epoch AI log |
| 38 | qwen3.8-max-0902_xhighAlibaba | closed | effort xhigh | 92.3%±1.7 | 2 Sep 2026 | Epoch AI log |
| 39 | moonshotai/Kimi-K3Epoch: kimi-k3_high, Moonshot | open | effort high | 91.9%±1.9 | 7 Aug 2026 | Epoch AI log |
| 40 | zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI) | open | effort max | 91.9%±1.6 | 24 Jun 2026 | Epoch AI log |
| 41 | deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_max, DeepSeek | open | effort max | 91.7%±1.5 | 18 Aug 2026 | Epoch AI log |
| 42 | gpt-5.6-luna_maxOpenAI | closed | effort max | 91.6%±1.7 | 9 Jul 2026 | Epoch AI log |
| 43 | gpt-5.2-2025-12-11_xhighOpenAI | closed | effort xhigh | 91.4%±1.8 | 13 Dec 2025 | Epoch AI log |
| 44 | claude-opus-4-8_maxAnthropic | closed | effort max | 91.0%±1.9 | 7 Jun 2026 | Epoch AI log |
| 45 | deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeek | open | effort max | 91.0%±1.8 | 2 Aug 2026 | Epoch AI log |
| 46 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_high, DeepSeek | open | effort high | 90.9%±2.0 | 6 Aug 2026 | Epoch AI log |
| 47 | zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI) | open | effort max | 90.9%±1.6 | 24 Aug 2026 | Epoch AI log |
| 48 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMax | open | default | 90.9%±2.0 | 10 Aug 2026 | Epoch AI log |
| 49 | qwen3.7-maxAlibaba | closed | default | 90.9%±2.0 | 7 Aug 2026 | Epoch AI log |
| 50 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 90.8%±1.7 | 1 May 2026 | Epoch AI log |
| 51 | gpt-5.5_lowOpenAI | closed | effort low | 90.7%±1.8 | 5 May 2026 | Epoch AI log |
| 52 | claude-opus-5-5_maxAnthropic | closed | effort max | 90.6%±1.9 | 22 Sep 2026 | Epoch AI log |
| 53 | claude-opus-4-6_32KAnthropic | closed | default | 90.5%±1.7 | 6 Feb 2026 | Epoch AI log |
| 54 | claude-sonnet-5_xhighAnthropic | closed | effort xhigh | 90.5%±1.8 | 1 Jul 2026 | Epoch AI log |
| 55 | gpt-6-luna_maxOpenAI | closed | effort max | 90.5%±1.9 | 22 Sep 2026 | Epoch AI log |
| 56 | claude-opus-4-7_xhighAnthropic | closed | effort xhigh | 90.2%±1.8 | 17 Apr 2026 | Epoch AI log |
| 57 | zai-org/GLM-5.3-FlashEpoch: glm-5.3-flash_max, Z.ai (Zhipu AI) | open | effort max | 90.2%±1.7 | 26 Aug 2026 | Epoch AI log |
| 58 | mistral-large-4-0-oss_highMistral AIno page match | open | effort high | 90.2%±1.8 | 2 Nov 2026 | Epoch AI log |
| 59 | zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI) | open | default | 89.9%±2.1 | 10 Aug 2026 | Epoch AI log |
| 60 | gpt-5.4-2026-03-05_highOpenAI | closed | effort high | 89.9%±2.1 | 15 Jul 2026 | Epoch AI log |
| 61 | gpt-5.6-sol_lowOpenAI | closed | effort low | 89.9%±2.1 | 7 Aug 2026 | Epoch AI log |
| 62 | deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_max, DeepSeek | open | effort max | 89.8%±1.7 | 9 Oct 2026 | Epoch AI log |
| 63 | muse-sparkMeta AI | closed | default | 89.8%±2.2 | 8 Apr 2026 | Epoch AI |
| 64 | claude-haiku-5-5_maxAnthropic | closed | effort max | 89.6%±2.0 | 9 Oct 2026 | Epoch AI log |
| 65 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeek | open | effort max | 89.6%±1.7 | 16 Jun 2026 | Epoch AI log |
| 66 | doubao-seed-2.1-turbo-260628_highByteDance | closed | effort high | 89.6%±1.8 | 30 Oct 2026 | Epoch AI log |
| 67 | gemini-3-flash-preview_highGoogle DeepMind | closed | effort high | 89.4%±2.2 | 6 Aug 2026 | Epoch AI log |
| 68 | doubao-seed-2.1-pro-260628_highByteDance | closed | effort high | 89.3%±1.8 | 28 Oct 2026 | Epoch AI log |
| 69 | grok-4.20-0309-reasoningxAI | closed | default | 89.3%±1.9 | 13 Jul 2026 | Epoch AI log |
| 70 | gemini-3.5-flash_lowGoogle DeepMind | closed | effort low | 88.9%±2.2 | 6 Aug 2026 | Epoch AI log |
| 71 | gpt-5.4-2026-03-05_mediumOpenAI | closed | effort medium | 88.9%±2.2 | 15 Jul 2026 | Epoch AI log |
| 72 | grok-4.3_highxAI | closed | effort high | 88.8%±2.0 | 17 Jun 2026 | Epoch AI log |
| 73 | claude-opus-4-6_64KAnthropic | closed | default | 88.8%±1.9 | 6 Feb 2026 | Epoch AI log |
| 74 | meta-models/Muse-Spark-Open-816B-A42BEpoch: Muse-Spark-Open-816B-A42B_xhigh, Meta AI | open | effort xhigh | 88.6%±1.9 | 15 Oct 2026 | Epoch AI log |
| 75 | thinkingmachines/Inkling-SmallEpoch: inkling-small_xhigh, Thinking Machines | open | effort xhigh | 88.5%±1.9 | 14 Aug 2026 | Epoch AI log |
| 76 | claude-opus-4-6_maxAnthropic | closed | effort max | 88.4%±2.3 | 6 Aug 2026 | Epoch AI log |
| 77 | claude-opus-4-8_lowAnthropic | closed | effort low | 88.4%±2.3 | 6 Aug 2026 | Epoch AI log |
| 78 | qwen3.6-plusAlibaba | closed | default | 88.4%±2.3 | 7 Aug 2026 | Epoch AI log |
| 79 | thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machines | open | effort xhigh | 88.3%±1.9 | 5 Aug 2026 | Epoch AI log |
| 80 | doubao-seed-2.0-pro-260215_highByteDance | closed | effort high | 88.2%±1.9 | 28 Oct 2026 | Epoch AI log |
| 81 | gpt-5.2-2025-12-11_highOpenAI | closed | effort high | 88.2%±1.9 | 11 Dec 2025 | Epoch AI log |
| 82 | doubao-seed-2.0-lite-260428_highByteDance | closed | effort high | 88.1%±1.9 | 28 Oct 2026 | Epoch AI log |
| 83 | mistral-large-4-0_highMistral AIno page match | open | effort high | 88.1%±1.9 | 12 Oct 2026 | Epoch AI log |
| 84 | claude-opus-5_lowAnthropic | closed | effort low | 87.9%±2.3 | 6 Aug 2026 | Epoch AI log |
| 85 | zai-org/GLM-5.2Epoch: glm-5.2_low, Z.ai (Zhipu AI) | open | effort low | 87.9%±2.3 | 10 Aug 2026 | Epoch AI log |
| 86 | gpt-5.2-2025-12-11_mediumOpenAI | closed | effort medium | 87.9%±1.9 | 11 Dec 2025 | Epoch AI log |
| 87 | moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, Moonshot | open | default | 87.9%±2.3 | 7 Aug 2026 | Epoch AI log |
| 88 | qwen3.7-plusAlibaba | closed | default | 87.9%±2.3 | 7 Aug 2026 | Epoch AI log |
| 89 | zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI) | open | default | 87.8%±2.3 | 12 Feb 2026 | Epoch AI log |
| 90 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 87.6%±1.9 | 13 Nov 2025 | Epoch AI log |
| 91 | moonshotai/Kimi-K2.5Epoch: fireworks/kimi-k2p5, Moonshot | open | default | 87.6%±1.9 | 2 Feb 2026 | Epoch AI log |
| 92 | claude-sonnet-4-6_32KAnthropic | closed | default | 87.4%±2.0 | 20 Feb 2026 | Epoch AI log |
| 93 | gpt-5.6-terra_lowOpenAI | closed | effort low | 87.4%±2.4 | 7 Aug 2026 | Epoch AI log |
| 94 | qwen3.6-max-previewAlibaba | not stated | default | 87.4%±2.4 | 7 Aug 2026 | Epoch AI log |
| 95 | grok-4-0709xAI | closed | default | 87.0%±2.0 | not dated | Epoch AI |
| 96 | gpt-5.4-mini-2026-03-17_xhighOpenAI | closed | effort xhigh | 86.9%±2.4 | 7 Aug 2026 | Epoch AI log |
| 97 | claude-opus-4-7_maxAnthropic | closed | effort max | 86.4%±2.4 | 6 Aug 2026 | Epoch AI log |
| 98 | gemini-3.5-flash_minimalGoogle DeepMind | closed | effort minimal | 86.4%±2.4 | 15 Jul 2026 | Epoch AI log |
| 99 | gemini-3.6-flash_lowGoogle DeepMind | closed | effort low | 86.4%±2.4 | 6 Aug 2026 | Epoch AI log |
| 100 | Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b_none, Alibaba | open | effort none | 86.4%±2.4 | 7 Aug 2026 | Epoch AI log |
| 101 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 86.2%±2.1 | 29 Oct 2025 | Epoch AI log |
| 102 | claude-opus-4-5-20251101_32KAnthropic | closed | default | 86.0%±2.1 | 24 Nov 2025 | Epoch AI log |
| 103 | claude-fable-5_maxAnthropic | closed | effort max | 85.9%±2.5 | 6 Aug 2026 | Epoch AI log |
| 104 | gemini-3.6-flash_minimalGoogle DeepMind | closed | effort minimal | 85.9%±2.5 | 7 Aug 2026 | Epoch AI log |
| 105 | Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibaba | open | default | 85.9%±2.5 | 7 Aug 2026 | Epoch AI log |
| 106 | Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibaba | open | default | 85.9%±2.5 | 7 Aug 2026 | Epoch AI log |
| 107 | claude-opus-4-5-20251101_16KAnthropic | closed | default | 85.5%±2.2 | 25 Nov 2025 | Epoch AI log |
| 108 | claude-opus-4-8_noneAnthropic | closed | effort none | 85.4%±2.5 | 6 Aug 2026 | Epoch AI log |
| 109 | gpt-5-2025-08-07_mediumOpenAI | closed | effort medium | 85.4%±2.1 | 7 Aug 2025 | Epoch AI log |
| 110 | nemotron-3-ultraNvidiano page match | open | default | 85.4%±2.5 | 10 Aug 2026 | Epoch AI log |
| 111 | gemini-2.5-proGoogle DeepMind | closed | default | 85.3%±2.1 | 16 Nov 2025 | Epoch AI log |
| 112 | gpt-5.1-2025-11-13_mediumOpenAI | closed | effort medium | 85.0%±2.1 | 17 Nov 2025 | Epoch AI log |
| 113 | gemini-2.5-pro-preview-06-05Google DeepMind | closed | default | 84.8%±2.6 | 5 Jun 2025 | Epoch AI log |
| 114 | gpt-5.4-2026-03-05_lowOpenAI | closed | effort low | 84.8%±2.6 | 15 Jul 2026 | Epoch AI log |
| 115 | moonshotai/Kimi-K3Epoch: kimi-k3_low, Moonshot | open | effort low | 84.8%±2.6 | 7 Aug 2026 | Epoch AI log |
| 116 | qwen3.5-plusAlibaba | not stated | default | 84.8%±2.6 | 7 Aug 2026 | Epoch AI log |
| 117 | Qwen/Qwen3.6-27BEpoch: qwen3.6-27b_none, Alibaba | open | effort none | 84.8%±2.6 | 7 Aug 2026 | Epoch AI log |
| 118 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b_none, Alibaba | open | effort none | 84.8%±2.6 | 7 Aug 2026 | Epoch AI log |
| 119 | kimi-k2-thinking-turboMoonshotno page match | open | default | 84.2%±2.1 | 11 Nov 2025 | Epoch AI log |
| 120 | gemini-2.5-pro-exp-03-25Google DeepMind | closed | default | 83.8%±2.6 | 31 Mar 2025 | Epoch AI log |
| 121 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibaba | open | default | 83.8%±2.6 | 7 Aug 2026 | Epoch AI log |
| 122 | gpt-5.4-mini-2026-03-17_highOpenAI | closed | effort high | 83.6%±2.2 | 15 Apr 2026 | Epoch AI log |
| 123 | Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibaba | open | default | 83.5%±2.2 | 28 Aug 2026 | Epoch AI log |
| 124 | deepseek-reasonerDeepSeekno page match | open | default | 83.4%±2.0 | 16 Dec 2025 | Epoch AI log |
| 125 | claude-fable-5_highAnthropic | closed | effort high | 83.3%±2.7 | 6 Aug 2026 | Epoch AI log |
| 126 | claude-sonnet-4-6_highAnthropic | closed | effort high | 83.3%±2.7 | 13 Jul 2026 | Epoch AI log |
| 127 | claude-sonnet-4-6_mediumAnthropic | closed | effort medium | 83.3%±2.7 | 13 Jul 2026 | Epoch AI log |
| 128 | gemini-3.5-flash-lite_highGoogle DeepMind | closed | effort high | 83.3%±2.7 | 6 Aug 2026 | Epoch AI log |
| 129 | zai-org/GLM-4.7Epoch: glm-4.7, Z.ai (Zhipu AI) | open | default | 83.3%±2.4 | 29 Jan 2026 | Epoch AI log |
| 130 | qwen3.6-flashAlibaba | not stated | default | 83.3%±2.7 | 7 Aug 2026 | Epoch AI log |
| 131 | gemini-3-flash-previewGoogle DeepMind | closed | default | 83.2%±1.9 | 17 Dec 2025 | Epoch AI log |
| 132 | gpt-5.6-sol_noneOpenAI | closed | effort none | 82.8%±2.7 | 7 Aug 2026 | Epoch AI log |
| 133 | gpt-5.2-2025-12-11_lowOpenAI | closed | effort low | 82.7%±2.3 | 11 Dec 2025 | Epoch AI log |
| 134 | gpt-5.5-instantOpenAI | closed | default | 82.5%±2.3 | 2 Aug 2026 | Epoch AI log |
| 135 | claude-sonnet-4-5-20250929_59KAnthropic | closed | default | 82.3%±2.7 | 28 Oct 2025 | Epoch AI log |
| 136 | gpt-5.6-luna_lowOpenAI | closed | effort low | 82.3%±2.7 | 7 Aug 2026 | Epoch AI log |
| 137 | qwen3.5-flashAlibaba | not stated | default | 82.3%±2.7 | 7 Aug 2026 | Epoch AI log |
| 138 | qwen3.7-flashAlibaba | closed | default | 82.3%±2.7 | 7 Aug 2026 | Epoch AI log |
| 139 | gemini-3.1-flash-lite_highGoogle | closed | effort high | 81.8%±2.7 | 6 Aug 2026 | Epoch AI log |
| 140 | o3-2025-04-16_highOpenAI | closed | effort high | 81.8%±2.1 | 16 Apr 2025 | Epoch AI log |
| 141 | qwen3.7-plus_noneAlibaba | closed | effort none | 81.8%±2.7 | 7 Aug 2026 | Epoch AI log |
| 142 | claude-sonnet-4-5-20250929_32KAnthropic | closed | default | 81.7%±2.8 | 21 Oct 2025 | Epoch AI log |
| 143 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3_none, MiniMax | open | effort none | 81.3%±2.8 | 10 Aug 2026 | Epoch AI log |
| 144 | Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b_none, Alibaba | open | effort none | 81.2%±2.3 | 28 Aug 2026 | Epoch AI log |
| 145 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 80.8%±2.8 | 7 Aug 2026 | Epoch AI log |
| 146 | qwen3.7-flash_noneAlibaba | closed | effort none | 80.8%±2.8 | 7 Aug 2026 | Epoch AI log |
| 147 | claude-opus-4-5-20251101Anthropic | closed | default | 80.7%±2.4 | 24 Nov 2025 | Epoch AI log |
| 148 | claude-sonnet-5_maxAnthropic | closed | effort max | 80.3%±2.8 | 6 Aug 2026 | Epoch AI log |
| 149 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 80.1%±2.6 | 11 Dec 2025 | Epoch AI log |
| 150 | o3-2025-04-16_lowOpenAI | closed | effort low | 79.8%±2.9 | 15 Jul 2026 | Epoch AI log |
| 151 | claude-3-7-sonnet-20250219_64KAnthropic | closed | default | 79.7%±2.7 | 26 May 2025 | Epoch AI log |
| 152 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 79.6%±2.4 | 16 Apr 2025 | Epoch AI log |
| 153 | claude-sonnet-4-20250514_59KAnthropic | closed | default | 79.2%±2.7 | 26 May 2025 | Epoch AI log |
| 154 | Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibaba | open | default | 79.0%±2.4 | 27 Aug 2026 | Epoch AI log |
| 155 | Qwen/Qwen3.5-9BEpoch: qwen3.5-9B_none, Alibaba | open | effort none | 78.9%±2.4 | 27 Aug 2026 | Epoch AI log |
| 156 | claude-fable-5_lowAnthropic | closed | effort low | 78.8%±2.9 | 6 Aug 2026 | Epoch AI log |
| 157 | claude-sonnet-4-5-20250929_16KAnthropic | closed | default | 78.8%±2.9 | 28 Oct 2025 | Epoch AI log |
| 158 | claude-sonnet-4-6_maxAnthropic | closed | effort max | 78.8%±2.9 | 6 Aug 2026 | Epoch AI log |
| 159 | gpt-5.4-nano-2026-03-17_highOpenAI | closed | effort high | 78.5%±2.4 | 14 Apr 2026 | Epoch AI log |
| 160 | claude-sonnet-4-20250514_32KAnthropic | closed | default | 78.3%±2.9 | 22 May 2025 | Epoch AI log |
| 161 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 77.8%±3.0 | 7 Aug 2026 | Epoch AI log |
| 162 | seed-1.6ByteDance | closed | default | 77.6%±2.5 | 30 Oct 2026 | Epoch AI log |
| 163 | claude-opus-4-1-20250805_16KAnthropic | closed | default | 77.3%±3.0 | 5 Aug 2025 | Epoch AI log |
| 164 | gpt-5.5_noneOpenAI | closed | effort none | 77.3%±3.0 | 7 Aug 2026 | Epoch AI log |
| 165 | gpt-5.6-terra_noneOpenAI | closed | effort none | 77.3%±3.0 | 7 Aug 2026 | Epoch AI log |
| 166 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 77.0%±2.6 | 13 Feb 2025 | Epoch AI log |
| 167 | claude-3-7-sonnet-20250219_16KAnthropic | closed | default | 76.8%±3.0 | 26 Feb 2025 | Epoch AI log |
| 168 | claude-3-7-sonnet-20250219_32KAnthropic | closed | default | 76.8%±3.0 | 10 Mar 2025 | Epoch AI log |
| 169 | claude-opus-4-1-20250805_27KAnthropic | closed | default | 76.8%±3.0 | 5 Aug 2025 | Epoch AI log |
| 170 | o1-2024-12-17_highOpenAI | closed | effort high | 76.8%±3.0 | 13 Feb 2025 | Epoch AI log |
| 171 | deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeek | open | default | 76.3%±2.4 | 29 May 2025 | Epoch AI log |
| 172 | claude-opus-4-20250514_16KAnthropic | closed | default | 76.3%±3.0 | 22 May 2025 | Epoch AI log |
| 173 | grok-3-mini-beta_lowxAI | closed | effort low | 76.3%±3.0 | 10 Apr 2025 | Epoch AI log |
| 174 | claude-sonnet-4-20250514_16KAnthropic | closed | default | 75.8%±3.1 | 22 May 2025 | Epoch AI log |
| 175 | gemini-3.5-flash-lite_lowGoogle DeepMind | closed | effort low | 75.8%±3.1 | 6 Aug 2026 | Epoch AI log |
| 176 | google/gemma-4-31B-itEpoch: gemma-4-31b-it_minimal, Google DeepMind | open | effort minimal | 75.8%±3.1 | 6 Aug 2026 | Epoch AI log |
| 177 | grok-3-betaxAI | closed | default | 75.8%±2.7 | 26 May 2025 | Epoch AI log |
| 178 | o1-2024-12-17_mediumOpenAI | closed | effort medium | 75.8%±3.1 | 27 Jan 2025 | Epoch AI log |
| 179 | openai/gpt-oss-120bEpoch: openai/gpt-oss-120b_high, OpenAI | open | effort high | 75.8%±2.7 | 11 Dec 2025 | Epoch AI log |
| 180 | grok-3-mini-beta_highxAI | closed | effort high | 75.5%±2.8 | 26 May 2025 | Epoch AI log |
| 181 | o4-mini-2025-04-16_lowOpenAI | closed | effort low | 75.3%±3.1 | 11 Jul 2026 | Epoch AI log |
| 182 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 75.0%±2.3 | 30 Oct 2025 | Epoch AI log |
| 183 | gpt-5.4-2026-03-05_noneOpenAI | closed | effort none | 74.7%±3.1 | 15 Jul 2026 | Epoch AI log |
| 184 | o3-mini-2025-01-31_mediumOpenAI | closed | effort medium | 74.3%±2.5 | 31 Jan 2025 | Epoch AI log |
| 185 | gemini-3.1-flash-lite_lowGoogle | closed | effort low | 74.2%±3.1 | 6 Aug 2026 | Epoch AI log |
| 186 | gemini-3.5-flash-lite_minimalGoogle DeepMind | closed | effort minimal | 74.2%±3.1 | 6 Aug 2026 | Epoch AI log |
| 187 | o1-2024-12-17_lowOpenAI | closed | effort low | 74.2%±3.1 | 20 Jul 2026 | Epoch AI log |
| 188 | claude-sonnet-4-5-20250929Anthropic | closed | default | 73.7%±3.1 | 29 Sep 2025 | Epoch AI log |
| 189 | gemini-3.1-flash-lite_minimalGoogle | closed | effort minimal | 73.7%±3.1 | 6 Aug 2026 | Epoch AI log |
| 190 | claude-opus-4-1-20250805Anthropic | closed | default | 73.2%±3.2 | 5 Aug 2025 | Epoch AI log |
| 191 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_none, DeepSeek | open | effort none | 73.2%±3.2 | 6 Aug 2026 | Epoch AI log |
| 192 | google/gemma-4-26B-A4B-itEpoch: gemma-4-26b-a4b-it_minimal, Google DeepMind | open | effort minimal | 73.2%±3.2 | 6 Aug 2026 | Epoch AI log |
| 193 | gpt-5.2-2025-12-11_noneOpenAI | closed | effort none | 73.2%±3.2 | 13 Jul 2026 | Epoch AI log |
| 194 | qwen3-max-2025-09-23Alibaba | closed | default | 72.6%±2.7 | 6 Oct 2025 | Epoch AI log |
| 195 | gpt-5.4-nano-2026-03-17_lowOpenAI | closed | effort low | 72.2%±3.2 | 7 Aug 2026 | Epoch AI log |
| 196 | deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeek | open | default | 71.7%±3.1 | 26 May 2025 | Epoch AI log |
| 197 | gpt-5-2025-08-07_minimalOpenAI | closed | effort minimal | 71.7%±3.2 | 20 Jul 2026 | Epoch AI log |
| 198 | gpt-5-mini-2025-08-07_minimalOpenAI | closed | effort minimal | 71.7%±3.2 | 7 Aug 2026 | Epoch AI log |
| 199 | gpt-5-mini-2025-08-07_mediumOpenAI | closed | effort medium | 71.7%±2.3 | 7 Aug 2025 | Epoch AI log |
| 200 | ByteDance-Seed/Seed-OSS-36B-InstructEpoch: seed-oss-36b-instruct | open | default | 71.5%±2.7 | 30 Aug 2026 | Epoch AI log |
| 201 | claude-haiku-4-5-20251001_32KAnthropic | closed | default | 71.2%±3.2 | 22 Oct 2025 | Epoch AI log |
| 202 | deepseek-chatDeepSeekno page match | open | default | 71.2%±3.2 | 16 Jul 2026 | Epoch AI log |
| 203 | zai-org/GLM-5.2Epoch: glm-5.2_none, Z.ai (Zhipu AI) | open | effort none | 71.2%±3.2 | 10 Aug 2026 | Epoch AI log |
| 204 | Qwen/Qwen3-235B-A22BEpoch: qwen3-235b-a22b, Alibaba | open | default | 70.7%±2.7 | 3 Jun 2025 | Epoch AI log |
| 205 | Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibaba | open | default | 70.1%±2.8 | 30 Aug 2026 | Epoch AI log |
| 206 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 69.4%±2.8 | 30 Oct 2025 | Epoch AI log |
| 207 | claude-opus-4-20250514Anthropic | closed | default | 69.2%±3.3 | 22 May 2025 | Epoch AI log |
| 208 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 68.7%±3.3 | 28 Feb 2025 | Epoch AI log |
| 209 | o3-mini-2025-01-31_lowOpenAI | closed | effort low | 68.2%±3.3 | 20 Jul 2026 | Epoch AI log |
| 210 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 67.6%±2.7 | 1 Apr 2025 | Epoch AI log |
| 211 | gpt-5-nano-2025-08-07_mediumOpenAI | closed | effort medium | 67.4%±2.7 | 7 Aug 2025 | Epoch AI log |
| 212 | meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AI | open | default | 67.0%±2.8 | 8 Apr 2025 | Epoch AI log |
| 213 | gpt-4.1-2025-04-14OpenAI | closed | default | 66.9%±2.8 | 14 Apr 2025 | Epoch AI log |
| 214 | claude-sonnet-4-20250514Anthropic | closed | default | 66.7%±3.4 | 22 May 2025 | Epoch AI log |
| 215 | gemini-2.5-pro-preview-05-06Google DeepMind | closed | default | 66.7%±3.4 | 3 Jun 2025 | Epoch AI log |
| 216 | gpt-5.1-2025-11-13_noneOpenAI | closed | effort none | 66.7%±3.4 | 7 Aug 2026 | Epoch AI log |
| 217 | claude-3-7-sonnet-20250219Anthropic | closed | default | 66.0%±2.7 | 24 Feb 2025 | Epoch AI log |
| 218 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 65.8%±2.7 | 14 Apr 2025 | Epoch AI log |
| 219 | Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibaba | open | default | 65.7%±2.8 | 28 Aug 2026 | Epoch AI log |
| 220 | gemini-2.0-pro-exp-02-05Google DeepMind | closed | default | 65.7%±3.4 | 7 Feb 2025 | Epoch AI log |
| 221 | qwq-plusAlibaba | closed | default | 65.4%±2.8 | 11 Apr 2025 | Epoch AI log |
| 222 | Qwen/QwQ-32BEpoch: QwQ-32B, Alibaba | open | default | 65.3%±2.8 | 28 Aug 2026 | Epoch AI log |
| 223 | deepseek-ai/DeepSeek-R1-Distill-Qwen-32BEpoch: DeepSeek-R1-Distill-Qwen-32B, DeepSeek | open | default | 64.1%±2.8 | 28 Aug 2026 | Epoch AI log |
| 224 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 64.1%±2.7 | 6 Feb 2025 | Epoch AI log |
| 225 | gpt-5.4-mini-2026-03-17_noneOpenAI | closed | effort none | 64.1%±3.4 | 7 Aug 2026 | Epoch AI log |
| 226 | Qwen/Qwen3-14BEpoch: qwen3-14b, Alibaba | open | default | 63.8%±2.9 | 28 Aug 2026 | Epoch AI log |
| 227 | gpt-5.6-luna_noneOpenAI | closed | effort none | 63.6%±3.4 | 7 Aug 2026 | Epoch AI log |
| 228 | o1-mini-2024-09-12_highOpenAI | closed | effort high | 62.4%±2.7 | 13 Feb 2025 | Epoch AI log |
| 229 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibaba | open | default | 61.7%±3.0 | 30 Aug 2026 | Epoch AI log |
| 230 | openai/gpt-oss-20bEpoch: gpt-oss-20b_medium, OpenAI | open | effort medium | 60.8%±2.5 | 27 Aug 2026 | Epoch AI log |
| 231 | zai-org/GLM-4.7-FlashEpoch: glm-4.7-flash | open | default | 60.5%±2.9 | 30 Aug 2026 | Epoch AI log |
| 232 | claude-haiku-4-5-20251001Anthropic | closed | default | 60.5%±2.8 | 16 Oct 2025 | Epoch AI log |
| 233 | mistral-medium-2505Mistral AI | closed | default | 59.5%±2.8 | 7 May 2025 | Epoch AI log |
| 234 | o1-mini-2024-09-12_mediumOpenAI | closed | effort medium | 59.5%±2.8 | 27 Jan 2025 | Epoch AI log |
| 235 | gpt-5-nano-2025-08-07_lowOpenAI | closed | effort low | 57.6%±3.5 | 13 Jul 2026 | Epoch AI log |
| 236 | gemini-1.5-pro-002Google DeepMind | closed | default | 57.2%±2.8 | 27 Jan 2025 | Epoch AI log |
| 237 | gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Google | closed | default | 57.1%±3.5 | 6 Feb 2025 | Epoch AI log |
| 238 | Qwen/Qwen3-8BEpoch: qwen3-8b, Alibaba | open | default | 56.8%±2.9 | 27 Aug 2026 | Epoch AI log |
| 239 | deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeek | open | default | 56.5%±2.8 | 27 Jan 2025 | Epoch AI log |
| 240 | qwen-max-2025-01-25Alibaba | closed | default | 56.1%±2.8 | 1 Apr 2025 | Epoch AI log |
| 241 | mistralai/Magistral-Small-2506Epoch: magistral-small-2506, Mistral AI | open | default | 56.1%±3.5 | 6 Aug 2026 | Epoch AI log |
| 242 | microsoft/phi-4Epoch: phi-4, Microsoft Research | open | default | 56.1%±2.6 | 31 Jan 2025 | Epoch AI log |
| 243 | deepseek-ai/DeepSeek-R1-Distill-Llama-70BEpoch: DeepSeek-R1-Distill-Llama-70B, DeepSeek | open | default | 55.7%±3.0 | 10 Mar 2025 | Epoch AI log |
| 244 | Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibaba | open | default | 55.6%±2.9 | 30 Aug 2026 | Epoch AI log |
| 245 | gpt-5.4-nano-2026-03-17_noneOpenAI | closed | effort none | 55.6%±3.5 | 7 Aug 2026 | Epoch AI log |
| 246 | claude-3-5-sonnet-20241022Anthropic | closed | default | 55.3%±2.8 | 27 Jan 2025 | Epoch AI log |
| 247 | Qwen/Qwen3-32BEpoch: Qwen3-32B_none, Alibaba | open | effort none | 54.1%±2.8 | 30 Aug 2026 | Epoch AI log |
| 248 | claude-3-5-sonnet-20240620Anthropic | closed | default | 54.0%±2.8 | 27 Jan 2025 | Epoch AI log |
| 249 | grok-2-1212xAI | closed | default | 53.8%±2.7 | 27 Jan 2025 | Epoch AI log |
| 250 | Qwen/Qwen3-14BEpoch: qwen3-14b_none, Alibaba | open | effort none | 53.4%±2.8 | 28 Aug 2026 | Epoch AI log |
| 251 | openai/gpt-oss-20bEpoch: gpt-oss-20b_low, OpenAI | open | effort low | 53.2%±2.5 | 27 Aug 2026 | Epoch AI log |
| 252 | Qwen/Qwen3-4BEpoch: qwen3-4b, Alibaba | open | default | 52.3%±3.0 | 28 Aug 2026 | Epoch AI log |
| 253 | meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AI | open | default | 51.8%±3.2 | 8 Apr 2025 | Epoch AI log |
| 254 | mistral-large-2411Mistral AIno page match | open | default | 51.3%±2.7 | 25 Feb 2025 | Epoch AI log |
| 255 | meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AI | open | default | 50.9%±2.6 | 27 Jan 2025 | Epoch AI log |
| 256 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B_none, Alibaba | open | effort none | 50.4%±2.8 | 30 Aug 2026 | Epoch AI log |
| 257 | o1-preview-2024-09-12OpenAI | closed | default | 50.3%±2.8 | 27 Jan 2025 | Epoch AI log |
| 258 | gpt-4o-2024-08-06OpenAI | closed | default | 49.2%±2.6 | 27 Jan 2025 | Epoch AI log |
| 259 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 49.1%±2.7 | 27 Jan 2025 | Epoch AI log |
| 260 | mistral-small-3.2-2506Mistral AIno page match | open | default | 49.1%±2.8 | 28 Aug 2026 | Epoch AI log |
| 261 | mistral-large-2407Mistral AIno page match | open | default | 49.0%±2.6 | 27 Jan 2025 | Epoch AI log |
| 262 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 48.9%±2.5 | 14 Apr 2025 | Epoch AI log |
| 263 | gpt-4o-2024-05-13OpenAI | closed | default | 48.9%±2.6 | 27 Jan 2025 | Epoch AI log |
| 264 | gpt-5-nano-2025-08-07_minimalOpenAI | closed | effort minimal | 48.5%±3.6 | 13 Jul 2026 | Epoch AI log |
| 265 | qwen-plus-2025-01-25Alibaba | closed | default | 48.1%±2.7 | 7 Apr 2025 | Epoch AI log |
| 266 | gpt-4o-2024-11-20OpenAI | closed | default | 47.9%±2.6 | 5 Feb 2025 | Epoch AI log |
| 267 | google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMind | open | default | 47.7%±2.7 | 27 Aug 2026 | Epoch AI log |
| 268 | mistralai/Magistral-Small-2509Epoch: magistral-small-2509, Mistral AI | open | default | 47.6%±2.6 | 30 Aug 2026 | Epoch AI log |
| 269 | mistral-small-2503Mistral AIno page match | open | default | 47.5%±2.8 | 18 Mar 2025 | Epoch AI log |
| 270 | Qwen/Qwen3-8BEpoch: qwen3-8b_none, Alibaba | open | effort none | 47.5%±2.6 | 27 Aug 2026 | Epoch AI log |
| 271 | meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AI | open | default | 47.4%±2.9 | 27 Jan 2025 | Epoch AI log |
| 272 | gemini-1.5-flash-002Google DeepMind | closed | default | 47.3%±2.8 | 27 Jan 2025 | Epoch AI log |
| 273 | mistral-small-3-2501Mistral AIno page match | open | default | 47.3%±2.8 | 30 Aug 2026 | Epoch AI log |
| 274 | claude-3-opus-20240229Anthropic | closed | default | 47.2%±2.6 | 27 Jan 2025 | Epoch AI log |
| 275 | gpt-4-turbo-2024-04-09OpenAI | closed | default | 46.6%±2.7 | 27 Feb 2025 | Epoch AI log |
| 276 | allenai/Llama-3.1-Tulu-3-70B-DPOEpoch: Llama-3.1-Tulu-3-70B-DPO, Allen Institute for AI,University of Washington | open | default | 46.3%±2.4 | 27 Jan 2025 | Epoch AI log |
| 277 | Qwen/Qwen2.5-32B-InstructEpoch: qwen2.5-32b-instruct, Alibaba | open | default | 46.1%±2.6 | 30 Jan 2025 | Epoch AI log |
| 278 | openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAI | open | effort high | 46.0%±3.6 | 6 Aug 2026 | Epoch AI log |
| 279 | gemini-1.5-pro-001Google DeepMind | closed | default | 45.9%±2.6 | 27 Jan 2025 | Epoch AI log |
| 280 | Qwen/Qwen3-4B-Instruct-2507Epoch: qwen3-4b-instruct-2507 | open | default | 45.8%±2.9 | 28 Aug 2026 | Epoch AI log |
| 281 | mistral-small-2501Mistral AIno page match | open | default | 45.3%±2.5 | 30 Jan 2025 | Epoch AI log |
| 282 | zai-org/GLM-4.7-FlashEpoch: glm-4.7-flash_none | open | effort none | 45.1%±2.2 | 30 Aug 2026 | Epoch AI log |
| 283 | deepseek-ai/DeepSeek-R1-Distill-Qwen-14BEpoch: DeepSeek-R1-Distill-Qwen-14B, DeepSeek | open | default | 44.7%±2.9 | 10 Mar 2025 | Epoch AI log |
| 284 | meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AI | open | default | 44.2%±2.5 | 27 Jan 2025 | Epoch AI log |
| 285 | Qwen/Qwen3-4BEpoch: qwen3-4b_none, Alibaba | open | effort none | 43.6%±2.7 | 28 Aug 2026 | Epoch AI log |
| 286 | WizardLM-2-8x22BMicrosoftno page match | open | default | 43.4%±2.3 | 27 Jan 2025 | Epoch AI log |
| 287 | gpt-4-1106-previewOpenAI | closed | default | 42.4%±2.4 | 27 Jan 2025 | Epoch AI log |
| 288 | gpt-4-0125-previewOpenAI | closed | default | 42.3%±2.7 | 27 Jan 2025 | Epoch AI log |
| 289 | mistral-small-3.1-2503Mistral AIno page match | open | default | 41.9%±2.3 | 30 Aug 2026 | Epoch AI log |
| 290 | qwen-turbo-2024-11-01Alibaba | closed | default | 41.8%±2.6 | 7 Apr 2025 | Epoch AI log |
| 291 | meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AI | open | default | 41.0%±2.3 | 27 Jan 2025 | Epoch AI log |
| 292 | Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, Alibaba | open | default | 40.8%±2.6 | 27 Jan 2025 | Epoch AI log |
| 293 | claude-3-sonnet-20240229Anthropic | closed | default | 40.6%±2.4 | 27 Jan 2025 | Epoch AI log |
| 294 | meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AI | open | default | 40.6%±2.6 | 27 Jan 2025 | Epoch AI log |
| 295 | gemini-1.5-flash-001Google DeepMind | closed | default | 40.4%±2.4 | 27 Jan 2025 | Epoch AI log |
| 296 | google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMind | open | default | 39.5%±2.5 | 28 Aug 2026 | Epoch AI log |
| 297 | mistral-large-2402Mistral AI | closed | default | 38.8%±2.5 | 27 Jan 2025 | Epoch AI log |
| 298 | claude-3-5-haiku-20241022Anthropic | closed | default | 38.1%±2.6 | 12 Mar 2025 | Epoch AI log |
| 299 | Qwen/Qwen3-1.7BEpoch: qwen3-1.7b, Alibaba | open | default | 38.0%±2.7 | 28 Aug 2026 | Epoch AI log |
| 300 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 37.7%±2.4 | 27 Jan 2025 | Epoch AI log |
| 301 | Hermes-2-Theta-Llama-3-70BNous Research,Arcee AIno page match | open | default | 37.5%±2.5 | 27 Jan 2025 | Epoch AI log |
| 302 | google/gemma-2-27b-itEpoch: gemma-2-27b-it, Google DeepMind | open | default | 36.5%±2.4 | 27 Jan 2025 | Epoch AI log |
| 303 | claude-3-haiku-20240307Anthropic | closed | default | 36.3%±2.4 | 27 Jan 2025 | Epoch AI log |
| 304 | gpt-4-0314OpenAI | closed | default | 35.7%±2.4 | 23 Oct 2025 | Epoch AI log |
| 305 | Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibaba | open | default | 35.5%±2.3 | 30 Aug 2026 | Epoch AI log |
| 306 | claude-2.0Anthropic | closed | default | 34.7%±2.5 | 27 Jan 2025 | Epoch AI log |
| 307 | open-mixtral-8x22bMistral AIno page match | open | default | 34.1%±2.1 | 27 Jan 2025 | Epoch AI log |
| 308 | gemini-1.0-pro-001Google DeepMind | closed | default | 34.0%±2.0 | 27 Jan 2025 | Epoch AI log |
| 309 | Eurus-2-7B-PRIMETsinghua University,University of Illinois Urbana-Champaign (UIUC),Shanghai AI Lab,Peking University,Shanghai Jiao Tong University,CUHK Shenzhen Research Instituteno page match | open | default | 33.9%±2.1 | 27 Jan 2025 | Epoch AI log |
| 310 | deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5BEpoch: DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek | open | default | 33.6%±2.1 | 28 Aug 2026 | Epoch AI log |
| 311 | claude-2.1Anthropic | closed | default | 33.0%±2.3 | 27 Jan 2025 | Epoch AI log |
| 312 | gemini-1.5-flash-8b-001Google DeepMind | closed | default | 33.0%±2.4 | 12 Mar 2025 | Epoch AI log |
| 313 | dbrx-instructDatabricksno page match | open | default | 32.9%±3.3 | 27 Jan 2025 | Epoch AI log |
| 314 | 01-ai/Yi-1.5-34B-ChatEpoch: Yi-1.5-34B-Chat, 01.AI | open | default | 32.0%±2.0 | 27 Jan 2025 | Epoch AI log |
| 315 | Qwen/Qwen1.5-32B-ChatEpoch: qwen1.5-32b-chat, Alibaba | open | default | 30.7%±2.0 | 27 Jan 2025 | Epoch AI log |
| 316 | gpt-4-0613OpenAI | closed | default | 30.7%±2.3 | 27 Jan 2025 | Epoch AI log |
| 317 | Qwen/Qwen3-1.7BEpoch: qwen3-1.7b_none, Alibaba | open | effort none | 30.6%±2.3 | 28 Aug 2026 | Epoch AI log |
| 318 | mistralai/Mixtral-8x7B-Instruct-v0.1Epoch: Mixtral-8x7B-Instruct-v0.1, Mistral AI | open | default | 30.6%±1.9 | 27 Jan 2025 | Epoch AI log |
| 319 | open-mistral-nemo-2407Mistral AIno page match | open | default | 29.9%±2.1 | 27 Jan 2025 | Epoch AI log |
| 320 | open-mixtral-8x7bMistral AIno page match | open | default | 29.8%±1.9 | 27 Jan 2025 | Epoch AI log |
| 321 | Qwen/Qwen1.5-72B-ChatEpoch: qwen1.5-72b-chat, Alibaba | open | default | 28.8%±2.0 | 27 Jan 2025 | Epoch AI log |
| 322 | ibm-granite/granite-4.0-microEpoch: granite-4.0-micro | open | default | 28.3%±1.8 | 30 Aug 2026 | Epoch AI log |
| 323 | gpt-3.5-turbo-1106OpenAI | closed | default | 28.0%±1.9 | 27 Jan 2025 | Epoch AI log |
| 324 | microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, Microsoft | open | default | 27.6%±1.6 | 31 Jan 2025 | Epoch AI log |
| 325 | google/gemma-2-9b-itEpoch: gemma-2-9b-it, Google DeepMind | open | default | 27.5%±2.3 | 27 Jan 2025 | Epoch AI log |
| 326 | gpt-3.5-turbo-0125OpenAI | closed | default | 27.2%±2.0 | 27 Jan 2025 | Epoch AI log |
| 327 | ministral-8b-2410Mistral AIno page match | open | default | 27.1%±2.1 | 27 Jan 2025 | Epoch AI log |
| 328 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | default | 27.0%±1.9 | 27 Aug 2026 | Epoch AI log |
| 329 | meta-llama/Llama-2-70b-chat-hfEpoch: Llama-2-70b-chat-hf, Meta AI | open | default | 26.3%±2.0 | 27 Jan 2025 | Epoch AI log |
| 330 | meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AI | open | default | 26.1%±1.7 | 27 Jan 2025 | Epoch AI log |
| 331 | ministral-3b-2410Mistral AI | closed | default | 25.3%±1.9 | 27 Jan 2025 | Epoch AI log |
| 332 | deepseek-ai/deepseek-llm-67b-chatEpoch: deepseek-llm-67b-chat, DeepSeek | open | default | 24.6%±1.4 | 27 Jan 2025 | Epoch AI log |
| 333 | ibm-granite/granite-4.0-1bEpoch: granite-4.0-1b | open | default | 24.0%±1.6 | 28 Aug 2026 | Epoch AI log |
| 334 | meta-llama/Llama-3.2-1B-InstructEpoch: Llama-3.2-1B-Instruct, Meta AI | open | default | 23.9%±1.6 | 30 Aug 2026 | Epoch AI log |
| 335 | google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMind | open | default | 23.2%±1.9 | 28 Aug 2026 | Epoch AI log |
| 336 | google/gemma-3-1b-itEpoch: gemma-3-1b-it, Google DeepMind | open | default | 19.9%±1.5 | 28 Aug 2026 | Epoch AI log |
| 337 | mistralai/Mistral-7B-Instruct-v0.3Epoch: Mistral-7B-Instruct-v0.3, Mistral AI | open | default | 15.2%±1.0 | 27 Jan 2025 | Epoch AI log |
| 338 | 01-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AI | open | default | 14.7%±1.1 | 27 Jan 2025 | Epoch AI log |
| 339 | open-mistral-7bMistral AIno page match | open | default | 13.2%±1.0 | 27 Jan 2025 | Epoch AI log |
| 340 | ibm-granite/granite-4.0-350mEpoch: granite-4.0-350m | open | default | 11.2%±0.9 | 28 Aug 2026 | Epoch AI log |
| 341 | deepseek-ai/DeepSeek-R1-0528-Qwen3-8BEpoch: deepseek-r1-0528-qwen3-8b | open | default | 9.3%±1.1 | 28 Aug 2026 | Epoch AI log |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method