OTIS Mock AIME 2024-2025
Re-run by Epoch AIMock exams in the style of the AIME, a US mathematics competition, where every answer is a whole number.
- Kind
- Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
- Score shown
- Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.1%. Error bars as the source gives them.
- Results
- 318 in all: 113 for open-weight models, 201 for closed ones, 4 where Epoch does not say. 99 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K3.1-Pro, 100.0%, number 16 of 318 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-fable-5_highAnthropic | closed | effort high | 100.0% | 6 Aug 2026 | Epoch AI log |
| 2 | claude-fable-5-1_maxAnthropic | closed | effort max | 100.0% | 1 Sep 2026 | Epoch AI log |
| 3 | claude-opus-5-5_maxAnthropic | closed | effort max | 100.0% | 22 Sep 2026 | Epoch AI log |
| 4 | claude-sonnet-5-5_maxAnthropic | closed | effort max | 100.0% | 29 Sep 2026 | Epoch AI log |
| 5 | gemini-4-argon_highGoogle DeepMind | closed | effort high | 100.0% | 13 Oct 2026 | Epoch AI log |
| 6 | gemini-4-carbon_highGoogle DeepMind | closed | effort high | 100.0% | 2 Nov 2026 | Epoch AI |
| 7 | gemini-4-hydrogen_xhighGoogle DeepMind | closed | effort xhigh | 100.0% | 13 Nov 2026 | Epoch AI log |
| 8 | gpt-5.5-pre-release_xhighOpenAI | closed | effort xhigh | 100.0% | 24 Apr 2026 | Epoch AI log |
| 9 | gpt-5.5-pro-pre-release_xhighOpenAI | closed | effort xhigh | 100.0% | 24 Apr 2026 | Epoch AI log |
| 10 | gpt-5.6-sol_maxOpenAI | closed | effort max | 100.0% | 9 Jul 2026 | Epoch AI log |
| 11 | gpt-6-astra_maxOpenAI | closed | effort max | 100.0% | 30 Aug 2026 | Epoch AI log |
| 12 | gpt-6-sol_maxOpenAI | closed | effort max | 100.0% | 22 Sep 2026 | Epoch AI log |
| 13 | gpt-6.1-sol_maxOpenAI | closed | effort max | 100.0% | 29 Sep 2026 | Epoch AI log |
| 14 | gpt-6.1-terra_maxOpenAI | closed | effort max | 100.0% | 26 Oct 2026 | Epoch AI log |
| 15 | gpt-6.2-astra_maxOpenAI | closed | effort max | 100.0% | 27 Oct 2026 | Epoch AI |
| 16 | moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro-together_max, Moonshot | open | effort max | 100.0% | 6 Nov 2026 | Epoch AI log |
| 17 | qwen3.8-max-0902_xhighAlibaba | closed | effort xhigh | 100.0% | 2 Sep 2026 | Epoch AI log |
| 18 | claude-fable-5_maxAnthropic | closed | effort max | 99.7%±0.3 | 10 Jun 2026 | Epoch AI log |
| 19 | claude-fable-5-2_maxAnthropic | closed | effort max | 99.7%±0.3 | 6 Nov 2026 | Epoch AI |
| 20 | gpt-5.6-terra_maxOpenAI | closed | effort max | 99.7%±0.3 | 9 Jul 2026 | Epoch AI log |
| 21 | deepseek-ai/DeepSeek-V4.1-Flash-1017Epoch: deepseek-v4.1-flash-1017_max, DeepSeek | open | effort max | 99.4%±0.4 | 19 Oct 2026 | Epoch AI log |
| 22 | mistral-large-4-0-oss_highMistral AIno page match | open | effort high | 99.4%±0.4 | 2 Nov 2026 | Epoch AI log |
| 23 | muse-wonder-1.0_maxMeta AI | closed | effort max | 99.4%±0.4 | 12 Oct 2026 | Epoch AI log |
| 24 | qwen3.8-max_xhighAlibaba | closed | effort xhigh | 99.4%±0.4 | 4 Aug 2026 | Epoch AI log |
| 25 | doubao-seed-evolving_highByteDance | closed | effort high | 99.2%±0.5 | 2 Nov 2026 | Epoch AI log |
| 26 | grok-4.6_xhighxAI | closed | effort xhigh | 99.2%±0.5 | 14 Aug 2026 | Epoch AI log |
| 27 | muse-spark-1.3_xhighMeta AI | closed | effort xhigh | 99.2%±0.6 | 17 Sep 2026 | Epoch AI log |
| 28 | claude-haiku-5-5_xhighAnthropic | closed | effort xhigh | 98.9%±0.5 | 7 Oct 2026 | Epoch AI log |
| 29 | claude-opus-5_maxAnthropic | closed | effort max | 98.9%±1.1 | 24 Jul 2026 | Epoch AI log |
| 30 | gemini-3.8-flash_highGoogle DeepMind | closed | effort high | 98.9%±0.9 | 2 Sep 2026 | Epoch AI log |
| 31 | gpt-6-luna_maxOpenAI | closed | effort max | 98.9%±0.7 | 22 Sep 2026 | Epoch AI log |
| 32 | deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_max, DeepSeek | open | effort max | 98.6%±1.0 | 18 Aug 2026 | Epoch AI log |
| 33 | claude-opus-4-8_maxAnthropic | closed | effort max | 98.3%±1.4 | 7 Jun 2026 | Epoch AI log |
| 34 | deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_max, DeepSeek | open | effort max | 98.3%±0.6 | 8 Oct 2026 | Epoch AI log |
| 35 | gpt-5.6-luna_maxOpenAI | closed | effort max | 98.3%±1.2 | 9 Jul 2026 | Epoch AI log |
| 36 | mistral-large-4-0_highMistral AIno page match | open | effort high | 98.3%±0.6 | 12 Oct 2026 | Epoch AI log |
| 37 | grok-4.7_xhighxAI | closed | effort xhigh | 98.1%±1.4 | 22 Sep 2026 | Epoch AI log |
| 38 | meta-models/Muse-Spark-Open-816B-A42BEpoch: Muse-Spark-Open-816B-A42B_xhigh, Meta AI | open | effort xhigh | 98.1%±1.3 | 15 Oct 2026 | Epoch AI log |
| 39 | claude-opus-4-7_xhighAnthropic | closed | effort xhigh | 97.8%±2.2 | 17 Apr 2026 | Epoch AI log |
| 40 | claude-fable-5_lowAnthropic | closed | effort low | 97.8%±2.2 | 6 Aug 2026 | Epoch AI log |
| 41 | claude-opus-4-8_lowAnthropic | closed | effort low | 97.8%±2.2 | 6 Aug 2026 | Epoch AI log |
| 42 | claude-opus-5Anthropic | closed | default | 97.8%±2.2 | 6 Aug 2026 | Epoch AI log |
| 43 | gpt-5.4-2026-03-05_highOpenAI | closed | effort high | 97.8%±2.2 | 15 Jul 2026 | Epoch AI log |
| 44 | grok-4.5_highxAI | closed | effort high | 97.8%±1.3 | 8 Jul 2026 | Epoch AI log |
| 45 | grok-4.6_highxAI | closed | effort high | 97.8%±1.0 | 12 Aug 2026 | Epoch AI log |
| 46 | claude-haiku-5-5_highAnthropic | closed | effort high | 97.2%±1.3 | 7 Oct 2026 | Epoch AI log |
| 47 | gemini-3.7-flash_highGoogle DeepMind | closed | effort high | 97.2%±1.9 | 14 Aug 2026 | Epoch AI log |
| 48 | moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshot | open | effort max | 97.2%±1.1 | 16 Jul 2026 | Epoch AI log |
| 49 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeek | open | effort max | 96.7%±2.0 | 17 Jun 2026 | Epoch AI log |
| 50 | doubao-seed-2.1-turbo-260628_highByteDance | closed | effort high | 96.1%±2.5 | 30 Oct 2026 | Epoch AI log |
| 51 | gpt-5.2-2025-12-11_highOpenAI | closed | effort high | 96.1%±2.6 | 11 Dec 2025 | Epoch AI log |
| 52 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 96.1%±2.4 | 2 May 2026 | Epoch AI log |
| 53 | gpt-5.2-2025-12-11_xhighOpenAI | closed | effort xhigh | 96.1%±2.7 | 13 Dec 2025 | Epoch AI log |
| 54 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 95.6%±3.1 | 20 Feb 2026 | Epoch AI log |
| 55 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_high, DeepSeek | open | effort high | 95.6%±3.1 | 6 Aug 2026 | Epoch AI log |
| 56 | doubao-seed-2.1-pro-260628_highByteDance | closed | effort high | 95.6%±2.9 | 30 Oct 2026 | Epoch AI log |
| 57 | gemini-3-flash-preview_highGoogle DeepMind | closed | effort high | 95.6%±3.1 | 6 Aug 2026 | Epoch AI log |
| 58 | gemini-3.1-pro-preview_highGoogle DeepMind | closed | effort high | 95.6%±3.1 | 6 Aug 2026 | Epoch AI log |
| 59 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 95.6%±2.7 | 25 May 2026 | Epoch AI log |
| 60 | gpt-5.4-2026-03-05_mediumOpenAI | closed | effort medium | 95.6%±3.1 | 15 Jul 2026 | Epoch AI log |
| 61 | gpt-5.6-sol_lowOpenAI | closed | effort low | 95.6%±3.1 | 7 Aug 2026 | Epoch AI log |
| 62 | moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, Moonshot | open | default | 95.6%±3.1 | 7 Aug 2026 | Epoch AI log |
| 63 | qwen3.7-maxAlibaba | closed | default | 95.6%±3.1 | 7 Aug 2026 | Epoch AI log |
| 64 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 95.3%±3.2 | 6 Mar 2026 | Epoch AI log |
| 65 | claude-sonnet-5_xhighAnthropic | closed | effort xhigh | 94.7%±2.3 | 1 Jul 2026 | Epoch AI log |
| 66 | claude-opus-4-6_64KAnthropic | closed | default | 94.4%±2.8 | 6 Feb 2026 | Epoch AI log |
| 67 | deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeek | open | effort max | 94.4%±2.1 | 2 Aug 2026 | Epoch AI log |
| 68 | gemini-3.6-flash_highGoogle DeepMind | closed | effort high | 94.2%±3.1 | 2 Aug 2026 | Epoch AI log |
| 69 | doubao-seed-2.0-pro-260215_highByteDance | closed | effort high | 93.9%±2.7 | 30 Oct 2026 | Epoch AI log |
| 70 | zai-org/GLM-5.3-FlashEpoch: glm-5.3-flash_max, Z.ai (Zhipu AI) | open | effort max | 93.9%±2.2 | 26 Aug 2026 | Epoch AI log |
| 71 | gpt-5.2-2025-12-11_mediumOpenAI | closed | effort medium | 93.9%±3.1 | 11 Dec 2025 | Epoch AI log |
| 72 | claude-opus-5_lowAnthropic | closed | effort low | 93.3%±3.8 | 6 Aug 2026 | Epoch AI log |
| 73 | zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI) | open | default | 93.3%±3.8 | 10 Aug 2026 | Epoch AI log |
| 74 | grok-4.3_highxAI | closed | effort high | 93.3%±3.0 | 17 Jun 2026 | Epoch AI log |
| 75 | moonshotai/Kimi-K3Epoch: kimi-k3_high, Moonshot | open | effort high | 93.3%±3.8 | 7 Aug 2026 | Epoch AI log |
| 76 | qwen3.6-plusAlibaba | closed | default | 93.3%±3.8 | 7 Aug 2026 | Epoch AI log |
| 77 | qwen3.7-plusAlibaba | closed | default | 93.3%±3.8 | 7 Aug 2026 | Epoch AI log |
| 78 | claude-opus-4-6_32KAnthropic | closed | default | 93.1%±3.3 | 6 Feb 2026 | Epoch AI log |
| 79 | gemini-3-flash-previewGoogle DeepMind | closed | default | 92.8%±3.5 | 17 Dec 2025 | Epoch AI log |
| 80 | grok-4.20-0309-reasoningxAI | closed | default | 92.2%±3.3 | 13 Jul 2026 | Epoch AI log |
| 81 | moonshotai/Kimi-K2.5Epoch: fireworks/kimi-k2p5, Moonshot | open | default | 92.2%±3.9 | 2 Feb 2026 | Epoch AI log |
| 82 | gemini-3-pro-previewGoogle DeepMind | closed | default | 91.4%±3.7 | 19 Nov 2025 | Epoch AI log |
| 83 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 91.4%±3.7 | 29 Oct 2025 | Epoch AI log |
| 84 | claude-opus-4-6_maxAnthropic | closed | effort max | 91.1%±4.3 | 6 Aug 2026 | Epoch AI log |
| 85 | doubao-seed-2.0-lite-260428_highByteDance | closed | effort high | 91.1%±3.6 | 30 Oct 2026 | Epoch AI log |
| 86 | zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI) | open | effort max | 91.1%±3.5 | 24 Aug 2026 | Epoch AI log |
| 87 | Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibaba | open | default | 91.1%±4.3 | 7 Aug 2026 | Epoch AI log |
| 88 | qwen3.6-max-previewAlibaba | not stated | default | 91.1%±4.3 | 7 Aug 2026 | Epoch AI log |
| 89 | thinkingmachines/Inkling-SmallEpoch: inkling-small_xhigh, Thinking Machines | open | effort xhigh | 90.0%±4.0 | 14 Aug 2026 | Epoch AI log |
| 90 | muse-sparkMeta AI | closed | default | 88.9%±4.7 | 8 Apr 2026 | Epoch AI |
| 91 | gemini-3.5-flash_lowGoogle DeepMind | closed | effort low | 88.9%±4.7 | 6 Aug 2026 | Epoch AI log |
| 92 | gpt-5.4-mini-2026-03-17_xhighOpenAI | closed | effort xhigh | 88.9%±4.7 | 7 Aug 2026 | Epoch AI log |
| 93 | gpt-5.6-terra_lowOpenAI | closed | effort low | 88.9%±4.7 | 7 Aug 2026 | Epoch AI log |
| 94 | thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machines | open | effort xhigh | 88.9%±3.9 | 5 Aug 2026 | Epoch AI log |
| 95 | openai/gpt-oss-120bEpoch: openai/gpt-oss-120b_high, OpenAI | open | effort high | 88.9%±4.5 | 11 Dec 2025 | Epoch AI log |
| 96 | Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibaba | open | default | 88.9%±4.7 | 7 Aug 2026 | Epoch AI log |
| 97 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 88.6%±4.0 | 13 Nov 2025 | Epoch AI log |
| 98 | deepseek-reasonerDeepSeekno page match | open | default | 87.8%±4.1 | 16 Dec 2025 | Epoch AI log |
| 99 | gpt-5.4-nano-2026-03-17_highOpenAI | closed | effort high | 87.8%±3.9 | 14 Apr 2026 | Epoch AI log |
| 100 | gpt-5-2025-08-07_mediumOpenAI | closed | effort medium | 87.2%±3.9 | 7 Aug 2025 | Epoch AI log |
| 101 | gpt-5.4-mini-2026-03-17_highOpenAI | closed | effort high | 87.2%±4.3 | 15 Apr 2026 | Epoch AI log |
| 102 | claude-opus-4-7_maxAnthropic | closed | effort max | 86.7%±5.1 | 6 Aug 2026 | Epoch AI log |
| 103 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 86.7%±4.3 | 30 Oct 2025 | Epoch AI log |
| 104 | nemotron-3-ultraNvidiano page match | open | default | 86.7%±5.1 | 10 Aug 2026 | Epoch AI log |
| 105 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 86.7%±5.1 | 10 Dec 2025 | Epoch AI log |
| 106 | qwen3.5-plusAlibaba | not stated | default | 86.7%±5.1 | 7 Aug 2026 | Epoch AI log |
| 107 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibaba | open | default | 86.7%±5.1 | 7 Aug 2026 | Epoch AI log |
| 108 | qwen3.7-flashAlibaba | closed | default | 86.7%±5.1 | 7 Aug 2026 | Epoch AI log |
| 109 | zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI) | open | effort max | 86.4%±4.2 | 25 Jun 2026 | Epoch AI log |
| 110 | claude-opus-4-5-20251101_32KAnthropic | closed | default | 86.1%±4.3 | 24 Nov 2025 | Epoch AI log |
| 111 | claude-sonnet-4-6_32KAnthropic | closed | default | 85.8%±4.5 | 20 Feb 2026 | Epoch AI log |
| 112 | gpt-5.1-2025-11-13_mediumOpenAI | closed | effort medium | 85.6%±4.4 | 17 Nov 2025 | Epoch AI log |
| 113 | gemini-2.5-proGoogle DeepMind | closed | default | 84.7%±4.8 | 16 Nov 2025 | Epoch AI log |
| 114 | claude-opus-4-8_noneAnthropic | closed | effort none | 84.4%±5.5 | 6 Aug 2026 | Epoch AI log |
| 115 | gpt-5.4-2026-03-05_lowOpenAI | closed | effort low | 84.4%±5.5 | 15 Jul 2026 | Epoch AI log |
| 116 | gpt-5.5_lowOpenAI | closed | effort low | 84.4%±5.5 | 7 Aug 2026 | Epoch AI log |
| 117 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 84.4%±5.5 | 7 Aug 2026 | Epoch AI log |
| 118 | qwen3.5-flashAlibaba | not stated | default | 84.4%±5.5 | 7 Aug 2026 | Epoch AI log |
| 119 | qwen3.6-flashAlibaba | not stated | default | 84.4%±5.5 | 7 Aug 2026 | Epoch AI log |
| 120 | grok-4-0709xAI | closed | default | 84.0%±5.0 | not dated | Epoch AI |
| 121 | o3-2025-04-16_highOpenAI | closed | effort high | 83.9%±4.4 | 16 Apr 2025 | Epoch AI log |
| 122 | zai-org/GLM-4.7Epoch: glm-4.7, Z.ai (Zhipu AI) | open | default | 83.3%±4.8 | 29 Jan 2026 | Epoch AI log |
| 123 | kimi-k2-thinking-turboMoonshotno page match | open | default | 83.1%±4.8 | 13 Nov 2025 | Epoch AI log |
| 124 | claude-sonnet-4-6_mediumAnthropic | closed | effort medium | 82.2%±5.8 | 13 Jul 2026 | Epoch AI log |
| 125 | gemini-3.6-flash_lowGoogle DeepMind | closed | effort low | 82.2%±5.8 | 7 Aug 2026 | Epoch AI log |
| 126 | google/gemma-4-26B-A4B-itEpoch: gemma-4-26b-a4b-it_minimal, Google DeepMind | open | effort minimal | 82.2%±5.8 | 28 Aug 2026 | Epoch AI log |
| 127 | Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b_none, Alibaba | open | effort none | 82.2%±5.8 | 7 Aug 2026 | Epoch AI log |
| 128 | claude-opus-4-5-20251101_16KAnthropic | closed | default | 81.7%±5.0 | 24 Nov 2025 | Epoch AI log |
| 129 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 81.7%±4.7 | 16 Apr 2025 | Epoch AI log |
| 130 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 81.1%±4.6 | 31 Oct 2025 | Epoch AI log |
| 131 | claude-sonnet-5_maxAnthropic | closed | effort max | 80.0%±6.0 | 6 Aug 2026 | Epoch AI log |
| 132 | gemini-3.1-flash-lite_highGoogle | closed | effort high | 80.0%±6.0 | 6 Aug 2026 | Epoch AI log |
| 133 | gemini-3.5-flash_minimalGoogle DeepMind | closed | effort minimal | 80.0%±6.0 | 15 Jul 2026 | Epoch AI log |
| 134 | gemini-3.6-flash_minimalGoogle DeepMind | closed | effort minimal | 80.0%±6.0 | 7 Aug 2026 | Epoch AI log |
| 135 | zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI) | open | default | 80.0%±6.0 | 12 Feb 2026 | Epoch AI log |
| 136 | qwen3.7-plus_noneAlibaba | closed | effort none | 80.0%±6.0 | 7 Aug 2026 | Epoch AI log |
| 137 | gpt-5.2-2025-12-11_lowOpenAI | closed | effort low | 78.9%±5.0 | 11 Dec 2025 | Epoch AI log |
| 138 | gpt-5-mini-2025-08-07_mediumOpenAI | closed | effort medium | 78.3%±4.8 | 7 Aug 2025 | Epoch AI log |
| 139 | claude-sonnet-4-5-20250929_32KAnthropic | closed | default | 77.8%±6.3 | 21 Oct 2025 | Epoch AI log |
| 140 | claude-sonnet-4-5-20250929_59KAnthropic | closed | default | 77.8%±6.3 | 28 Oct 2025 | Epoch AI log |
| 141 | grok-3-mini-beta_highxAI | closed | effort high | 77.8%±6.3 | 10 Apr 2025 | Epoch AI log |
| 142 | qwen3.7-flash_noneAlibaba | closed | effort none | 77.8%±6.3 | 7 Aug 2026 | Epoch AI log |
| 143 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 76.9%±5.2 | 27 Feb 2025 | Epoch AI log |
| 144 | claude-sonnet-4-6_highAnthropic | closed | effort high | 75.6%±6.5 | 13 Jul 2026 | Epoch AI log |
| 145 | zai-org/GLM-5.2Epoch: glm-5.2_low, Z.ai (Zhipu AI) | open | effort low | 75.6%±6.5 | 10 Aug 2026 | Epoch AI log |
| 146 | gpt-5-nano-2025-08-07_mediumOpenAI | closed | effort medium | 74.2%±5.2 | 7 Aug 2025 | Epoch AI log |
| 147 | google/gemma-4-31B-itEpoch: gemma-4-31b-it_minimal, Google DeepMind | open | effort minimal | 73.3%±6.7 | 6 Aug 2026 | Epoch AI log |
| 148 | o1-2024-12-17_highOpenAI | closed | effort high | 73.3%±6.7 | 15 Jul 2026 | Epoch AI log |
| 149 | o1-2024-12-17_mediumOpenAI | closed | effort medium | 73.3%±6.7 | 27 Feb 2025 | Epoch AI log |
| 150 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 73.3%±6.7 | 7 Aug 2026 | Epoch AI log |
| 151 | qwen3-max-2025-09-23Alibaba | closed | default | 73.3%±5.9 | 6 Oct 2025 | Epoch AI log |
| 152 | gemini-2.5-flash-preview-04-17Google DeepMind | closed | default | 73.1%±5.5 | 22 Apr 2025 | Epoch AI log |
| 153 | claude-sonnet-4-20250514_32KAnthropic | closed | default | 71.1%±6.8 | 22 May 2025 | Epoch AI log |
| 154 | claude-sonnet-4-5-20250929_16KAnthropic | closed | default | 71.1%±6.8 | 28 Oct 2025 | Epoch AI log |
| 155 | claude-sonnet-4-6_maxAnthropic | closed | effort max | 71.1%±6.8 | 6 Aug 2026 | Epoch AI log |
| 156 | gemini-3.5-flash-lite_highGoogle DeepMind | closed | effort high | 71.1%±6.8 | 6 Aug 2026 | Epoch AI log |
| 157 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMax | open | default | 71.1%±6.8 | 10 Aug 2026 | Epoch AI log |
| 158 | gemini-2.5-flash-preview-05-20Google DeepMind | closed | default | 70.8%±5.7 | 20 May 2025 | Epoch AI log |
| 159 | Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibaba | open | default | 70.3%±6.0 | 30 Aug 2026 | Epoch AI log |
| 160 | Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b_none, Alibaba | open | effort none | 70.0%±6.0 | 30 Aug 2026 | Epoch AI log |
| 161 | seed-1.6ByteDance | closed | default | 69.4%±6.1 | 30 Oct 2026 | Epoch AI log |
| 162 | claude-opus-4-1-20250805_27KAnthropic | closed | default | 68.9%±7.0 | 5 Aug 2025 | Epoch AI log |
| 163 | claude-sonnet-4-20250514_59KAnthropic | closed | default | 68.9%±7.0 | 23 May 2025 | Epoch AI log |
| 164 | gpt-5.4-nano-2026-03-17_lowOpenAI | closed | effort low | 68.9%±7.0 | 7 Aug 2026 | Epoch AI log |
| 165 | gpt-5.6-sol_noneOpenAI | closed | effort none | 68.9%±7.0 | 7 Aug 2026 | Epoch AI log |
| 166 | moonshotai/Kimi-K3Epoch: kimi-k3_low, Moonshot | open | effort low | 68.9%±7.0 | 7 Aug 2026 | Epoch AI log |
| 167 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b_none, Alibaba | open | effort none | 68.9%±7.0 | 7 Aug 2026 | Epoch AI log |
| 168 | gpt-5.5-instantOpenAI | closed | default | 68.1%±6.2 | 2 Aug 2026 | Epoch AI log |
| 169 | ByteDance-Seed/Seed-OSS-36B-InstructEpoch: seed-oss-36b-instruct | open | default | 67.5%±6.3 | 28 Aug 2026 | Epoch AI log |
| 170 | Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibaba | open | default | 66.9%±6.0 | 30 Aug 2026 | Epoch AI log |
| 171 | claude-haiku-4-5-20251001_32KAnthropic | closed | default | 66.7%±7.1 | 22 Oct 2025 | Epoch AI log |
| 172 | gpt-5.6-luna_lowOpenAI | closed | effort low | 66.7%±7.1 | 7 Aug 2026 | Epoch AI log |
| 173 | Qwen/Qwen3.6-27BEpoch: qwen3.6-27b_none, Alibaba | open | effort none | 66.7%±7.1 | 7 Aug 2026 | Epoch AI log |
| 174 | deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeek | open | default | 66.4%±4.9 | 29 May 2025 | Epoch AI log |
| 175 | Qwen/Qwen3-14BEpoch: qwen3-14b, Alibaba | open | default | 66.4%±6.2 | 28 Aug 2026 | Epoch AI log |
| 176 | openai/gpt-oss-20bEpoch: gpt-oss-20b_medium, OpenAI | open | effort medium | 65.3%±5.9 | 27 Aug 2026 | Epoch AI log |
| 177 | claude-opus-4-1-20250805_16KAnthropic | closed | default | 64.4%±7.2 | 5 Aug 2025 | Epoch AI log |
| 178 | claude-opus-4-20250514_27KAnthropic | closed | default | 64.4%±7.2 | 28 May 2025 | Epoch AI log |
| 179 | gpt-5.1-2025-11-13_lowOpenAI | closed | effort low | 63.9%±5.9 | 25 Nov 2025 | Epoch AI log |
| 180 | o3-mini-2025-01-31_mediumOpenAI | closed | effort medium | 63.9%±6.0 | 25 Feb 2025 | Epoch AI log |
| 181 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibaba | open | default | 62.8%±6.2 | 28 Aug 2026 | Epoch AI log |
| 182 | gpt-5.2-2025-12-11_noneOpenAI | closed | effort none | 62.2%±7.3 | 13 Jul 2026 | Epoch AI log |
| 183 | grok-3-mini-beta_lowxAI | closed | effort low | 62.2%±7.3 | 10 Apr 2025 | Epoch AI log |
| 184 | Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibaba | open | default | 62.2%±6.5 | 30 Aug 2026 | Epoch AI log |
| 185 | Qwen/Qwen3.5-9BEpoch: qwen3.5-9B_none, Alibaba | open | effort none | 61.7%±6.0 | 27 Aug 2026 | Epoch AI log |
| 186 | claude-opus-4-20250514_16KAnthropic | closed | default | 60.0%±7.4 | 22 May 2025 | Epoch AI log |
| 187 | gemini-3.5-flash-lite_lowGoogle DeepMind | closed | effort low | 60.0%±7.4 | 6 Aug 2026 | Epoch AI log |
| 188 | o3-2025-04-16_lowOpenAI | closed | effort low | 60.0%±7.4 | 15 Jul 2026 | Epoch AI log |
| 189 | Qwen/QwQ-32BEpoch: QwQ-32B, Alibaba | open | default | 59.2%±6.3 | 28 Aug 2026 | Epoch AI log |
| 190 | zai-org/GLM-4.7-FlashEpoch: glm-4.7-flash | open | default | 58.3%±6.6 | 30 Aug 2026 | Epoch AI log |
| 191 | claude-3-7-sonnet-20250219_64KAnthropic | closed | default | 57.8%±7.4 | 13 Mar 2025 | Epoch AI log |
| 192 | gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Google | closed | default | 57.8%±7.4 | 7 Mar 2025 | Epoch AI log |
| 193 | gpt-5.4-2026-03-05_noneOpenAI | closed | effort none | 57.8%±7.4 | 15 Jul 2026 | Epoch AI log |
| 194 | gpt-5.5_noneOpenAI | closed | effort none | 57.8%±7.4 | 7 Aug 2026 | Epoch AI log |
| 195 | o4-mini-2025-04-16_lowOpenAI | closed | effort low | 57.8%±7.4 | 13 Jul 2026 | Epoch AI log |
| 196 | Qwen/Qwen3-8BEpoch: qwen3-8b, Alibaba | open | default | 56.1%±6.5 | 27 Aug 2026 | Epoch AI log |
| 197 | Qwen/Qwen3.5-4BEpoch: qwen3.5-4b_none, Alibaba | open | effort none | 55.8%±6.2 | 28 Aug 2026 | Epoch AI log |
| 198 | deepseek-ai/DeepSeek-R1-Distill-Qwen-32BEpoch: DeepSeek-R1-Distill-Qwen-32B, DeepSeek | open | default | 55.6%±6.2 | 28 Aug 2026 | Epoch AI log |
| 199 | gpt-5-mini-2025-08-07_minimalOpenAI | closed | effort minimal | 55.6%±7.5 | 7 Aug 2026 | Epoch AI log |
| 200 | grok-3-betaxAI | closed | default | 55.6%±7.5 | 10 Apr 2025 | Epoch AI log |
| 201 | Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibaba | open | default | 54.4%±6.2 | 30 Aug 2026 | Epoch AI log |
| 202 | claude-3-7-sonnet-20250219_32KAnthropic | closed | default | 53.3%±7.5 | 12 Mar 2025 | Epoch AI log |
| 203 | claude-sonnet-4-20250514_16KAnthropic | closed | default | 53.3%±7.5 | 22 May 2025 | Epoch AI log |
| 204 | deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeek | open | default | 53.3%±7.5 | 26 Feb 2025 | Epoch AI log |
| 205 | gpt-5.6-terra_noneOpenAI | closed | effort none | 53.3%±7.5 | 7 Aug 2026 | Epoch AI log |
| 206 | o1-2024-12-17_lowOpenAI | closed | effort low | 53.3%±7.5 | 15 Jul 2026 | Epoch AI log |
| 207 | Qwen/Qwen3-4B-Instruct-2507Epoch: qwen3-4b-instruct-2507 | open | default | 52.2%±6.6 | 28 Aug 2026 | Epoch AI log |
| 208 | deepseek-ai/DeepSeek-R1-Distill-Llama-70BEpoch: DeepSeek-R1-Distill-Llama-70B, DeepSeek | open | default | 51.4%±6.1 | 7 Mar 2025 | Epoch AI log |
| 209 | gemini-3.5-flash-lite_minimalGoogle DeepMind | closed | effort minimal | 51.1%±7.5 | 6 Aug 2026 | Epoch AI log |
| 210 | openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAI | open | effort high | 50.8%±6.7 | 27 Aug 2026 | Epoch AI log |
| 211 | deepseek-ai/DeepSeek-R1-Distill-Qwen-14BEpoch: DeepSeek-R1-Distill-Qwen-14B, DeepSeek | open | default | 50.6%±6.0 | 28 Aug 2026 | Epoch AI log |
| 212 | deepseek-chatDeepSeekno page match | open | default | 48.9%±7.5 | 16 Jul 2026 | Epoch AI log |
| 213 | claude-opus-4-5-20251101Anthropic | closed | default | 48.1%±6.4 | 24 Nov 2025 | Epoch AI log |
| 214 | o1-mini-2024-09-12_highOpenAI | closed | effort high | 46.9%±6.5 | 6 Mar 2025 | Epoch AI log |
| 215 | claude-3-7-sonnet-20250219_16KAnthropic | closed | default | 46.7%±7.5 | 26 Feb 2025 | Epoch AI log |
| 216 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_none, DeepSeek | open | effort none | 46.7%±7.5 | 6 Aug 2026 | Epoch AI log |
| 217 | gpt-5-2025-08-07_minimalOpenAI | closed | effort minimal | 46.7%±7.5 | 20 Jul 2026 | Epoch AI log |
| 218 | gpt-5-nano-2025-08-07_lowOpenAI | closed | effort low | 46.7%±7.5 | 13 Jul 2026 | Epoch AI log |
| 219 | gpt-5.4-nano-2026-03-17_noneOpenAI | closed | effort none | 46.7%±7.5 | 7 Aug 2026 | Epoch AI log |
| 220 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 44.7%±6.1 | 14 Apr 2025 | Epoch AI log |
| 221 | o1-mini-2024-09-12_mediumOpenAI | closed | effort medium | 44.7%±6.3 | 6 Mar 2025 | Epoch AI log |
| 222 | gemini-3.1-flash-lite_lowGoogle | closed | effort low | 44.4%±7.5 | 6 Aug 2026 | Epoch AI log |
| 223 | o3-mini-2025-01-31_lowOpenAI | closed | effort low | 44.4%±7.5 | 20 Jul 2026 | Epoch AI log |
| 224 | deepseek-ai/DeepSeek-R1-0528-Qwen3-8BEpoch: deepseek-r1-0528-qwen3-8b | open | default | 43.9%±6.8 | 28 Aug 2026 | Epoch AI log |
| 225 | Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibaba | open | default | 42.8%±6.5 | 27 Aug 2026 | Epoch AI log |
| 226 | claude-opus-4-20250514Anthropic | closed | default | 42.2%±7.4 | 22 May 2025 | Epoch AI log |
| 227 | openai/gpt-oss-20bEpoch: gpt-oss-20b_low, OpenAI | open | effort low | 40.3%±6.3 | 27 Aug 2026 | Epoch AI log |
| 228 | claude-opus-4-1-20250805Anthropic | closed | default | 40.0%±7.4 | 5 Aug 2025 | Epoch AI log |
| 229 | gpt-5.6-luna_noneOpenAI | closed | effort none | 40.0%±7.4 | 7 Aug 2026 | Epoch AI log |
| 230 | gpt-4.1-2025-04-14OpenAI | closed | default | 38.3%±6.3 | 14 Apr 2025 | Epoch AI log |
| 231 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 37.8%±6.2 | 1 Apr 2025 | Epoch AI log |
| 232 | gemini-3.1-flash-lite_minimalGoogle | closed | effort minimal | 37.8%±7.3 | 6 Aug 2026 | Epoch AI log |
| 233 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 37.8%±7.3 | 28 Feb 2025 | Epoch AI log |
| 234 | gpt-5.1-2025-11-13_noneOpenAI | closed | effort none | 37.8%±7.3 | 7 Aug 2026 | Epoch AI log |
| 235 | claude-haiku-4-5-20251001Anthropic | closed | default | 35.8%±6.2 | 16 Oct 2025 | Epoch AI log |
| 236 | claude-sonnet-4-5-20250929Anthropic | closed | default | 35.6%±7.2 | 29 Sep 2025 | Epoch AI log |
| 237 | gpt-5-nano-2025-08-07_minimalOpenAI | closed | effort minimal | 35.6%±7.2 | 13 Jul 2026 | Epoch AI log |
| 238 | mistral-medium-2505Mistral AI | closed | default | 32.2%±6.0 | 7 May 2025 | Epoch AI log |
| 239 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 31.1%±6.3 | 25 Feb 2025 | Epoch AI log |
| 240 | o1-preview-2024-09-12OpenAI | closed | default | 31.1%±7.0 | 7 Mar 2025 | Epoch AI log |
| 241 | mistral-small-3.2-2506Mistral AIno page match | open | default | 30.3%±5.8 | 30 Aug 2026 | Epoch AI log |
| 242 | mistralai/Magistral-Small-2506Epoch: magistral-small-2506, Mistral AI | open | default | 30.0%±5.9 | 6 Aug 2026 | Epoch AI log |
| 243 | claude-sonnet-4-20250514Anthropic | closed | default | 28.9%±6.8 | 22 May 2025 | Epoch AI log |
| 244 | zai-org/GLM-5.2Epoch: glm-5.2_none, Z.ai (Zhipu AI) | open | effort none | 28.9%±6.8 | 10 Aug 2026 | Epoch AI log |
| 245 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 28.9%±6.0 | 14 Apr 2025 | Epoch AI log |
| 246 | mistralai/Magistral-Small-2509Epoch: magistral-small-2509, Mistral AI | open | default | 28.1%±5.6 | 30 Aug 2026 | Epoch AI log |
| 247 | gpt-5.4-mini-2026-03-17_noneOpenAI | closed | effort none | 26.7%±6.7 | 7 Aug 2026 | Epoch AI log |
| 248 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3_none, MiniMax | open | effort none | 26.7%±6.7 | 10 Aug 2026 | Epoch AI log |
| 249 | Qwen/Qwen3-14BEpoch: qwen3-14b_none, Alibaba | open | effort none | 25.8%±4.8 | 30 Aug 2026 | Epoch AI log |
| 250 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B_none, Alibaba | open | effort none | 25.6%±5.2 | 30 Aug 2026 | Epoch AI log |
| 251 | zai-org/GLM-4.7-FlashEpoch: glm-4.7-flash_none | open | effort none | 25.0%±4.9 | 28 Aug 2026 | Epoch AI log |
| 252 | gemini-1.5-pro-002Google DeepMind | closed | default | 23.1%±5.2 | 25 Feb 2025 | Epoch AI log |
| 253 | Qwen/Qwen3-32BEpoch: Qwen3-32B_none, Alibaba | open | effort none | 23.1%±5.1 | 30 Aug 2026 | Epoch AI log |
| 254 | google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMind | open | default | 22.5%±5.3 | 27 Aug 2026 | Epoch AI log |
| 255 | Qwen/Qwen3-8BEpoch: qwen3-8b_none, Alibaba | open | effort none | 22.2%±4.5 | 27 Aug 2026 | Epoch AI log |
| 256 | claude-3-7-sonnet-20250219Anthropic | closed | default | 21.9%±4.8 | 25 Feb 2025 | Epoch AI log |
| 257 | deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5BEpoch: DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek | open | default | 21.4%±5.3 | 28 Aug 2026 | Epoch AI log |
| 258 | meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8Epoch: Llama-4-Maverick-17B-128E-Instruct-FP8, Meta AI | open | default | 20.6%±4.9 | 8 Apr 2025 | Epoch AI log |
| 259 | qwen-plus-2025-01-25Alibaba | closed | default | 17.8%±4.4 | 7 Apr 2025 | Epoch AI log |
| 260 | google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMind | open | default | 16.7%±4.2 | 28 Aug 2026 | Epoch AI log |
| 261 | gemini-1.5-flash-002Google DeepMind | closed | default | 16.3%±4.1 | 25 Feb 2025 | Epoch AI log |
| 262 | qwen-max-2025-01-25Alibaba | closed | default | 16.1%±4.1 | 1 Apr 2025 | Epoch AI log |
| 263 | deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeek | open | default | 15.8%±4.3 | 25 Feb 2025 | Epoch AI log |
| 264 | microsoft/phi-4Epoch: phi-4, Microsoft Research | open | default | 13.8%±3.7 | 25 Feb 2025 | Epoch AI log |
| 265 | grok-2-1212xAI | closed | default | 11.5%±3.4 | 25 Feb 2025 | Epoch AI log |
| 266 | meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AI | open | default | 9.7%±3.2 | 25 Feb 2025 | Epoch AI log |
| 267 | claude-3-5-sonnet-20241022Anthropic | closed | default | 8.5%±2.6 | 25 Feb 2025 | Epoch AI log |
| 268 | mistral-large-2407Mistral AIno page match | open | default | 8.5%±2.5 | 25 Feb 2025 | Epoch AI log |
| 269 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 8.1%±2.6 | 25 Feb 2025 | Epoch AI log |
| 270 | Qwen/Qwen3-1.7BEpoch: qwen3-1.7b_none, Alibaba | open | effort none | 8.1%±2.8 | 28 Aug 2026 | Epoch AI log |
| 271 | meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AI | open | default | 7.8%±3.0 | 8 Apr 2025 | Epoch AI log |
| 272 | mistral-large-2411Mistral AIno page match | open | default | 7.8%±2.5 | 25 Feb 2025 | Epoch AI log |
| 273 | google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMind | open | default | 7.5%±2.8 | 28 Aug 2026 | Epoch AI log |
| 274 | Qwen/Qwen2.5-32B-InstructEpoch: qwen2.5-32b-instruct, Alibaba | open | default | 7.4%±2.1 | 25 Feb 2025 | Epoch AI log |
| 275 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 6.9%±2.7 | 30 Jul 2025 | Epoch AI log |
| 276 | gemini-1.5-pro-001Google DeepMind | closed | default | 6.8%±1.9 | 25 Feb 2025 | Epoch AI log |
| 277 | gpt-4-turbo-2024-04-09OpenAI | closed | default | 6.7%±2.4 | 27 Feb 2025 | Epoch AI log |
| 278 | mistral-small-3-2501Mistral AIno page match | open | default | 6.7%±2.7 | 30 Aug 2026 | Epoch AI log |
| 279 | claude-3-5-sonnet-20240620Anthropic | closed | default | 6.5%±2.2 | 25 Feb 2025 | Epoch AI log |
| 280 | gpt-4o-2024-08-06OpenAI | closed | default | 6.4%±2.6 | 25 Feb 2025 | Epoch AI log |
| 281 | gpt-4o-2024-05-13OpenAI | closed | default | 6.3%±2.3 | 25 Feb 2025 | Epoch AI log |
| 282 | gpt-4o-2024-11-20OpenAI | closed | default | 6.3%±2.4 | 25 Feb 2025 | Epoch AI log |
| 283 | qwen-turbo-2024-11-01Alibaba | closed | default | 6.1%±2.2 | 7 Apr 2025 | Epoch AI log |
| 284 | mistral-small-2503Mistral AIno page match | open | default | 5.8%±2.3 | 18 Mar 2025 | Epoch AI log |
| 285 | mistral-small-2501Mistral AIno page match | open | default | 5.3%±2.6 | 18 Mar 2025 | Epoch AI log |
| 286 | meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AI | open | default | 5.1%±2.4 | 25 Feb 2025 | Epoch AI log |
| 287 | claude-3-opus-20240229Anthropic | closed | default | 4.7%±2.0 | 25 Feb 2025 | Epoch AI log |
| 288 | gemini-1.5-flash-8b-001Google DeepMind | closed | default | 4.6%±2.0 | 25 Feb 2025 | Epoch AI log |
| 289 | allenai/Llama-3.1-Tulu-3-70B-DPOEpoch: Llama-3.1-Tulu-3-70B-DPO, Allen Institute for AI,University of Washington | open | default | 4.4%±1.8 | 7 Mar 2025 | Epoch AI log |
| 290 | claude-3-5-haiku-20241022Anthropic | closed | default | 4.3%±1.6 | 25 Feb 2025 | Epoch AI log |
| 291 | meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AI | open | default | 4.3%±2.2 | 25 Feb 2025 | Epoch AI log |
| 292 | gemini-1.5-flash-001Google DeepMind | closed | default | 3.9%±1.4 | 25 Feb 2025 | Epoch AI log |
| 293 | mistral-small-3.1-2503Mistral AIno page match | open | default | 3.9%±1.5 | 30 Aug 2026 | Epoch AI log |
| 294 | meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AI | open | default | 3.6%±1.7 | 25 Feb 2025 | Epoch AI log |
| 295 | ibm-granite/granite-4.0-microEpoch: granite-4.0-micro | open | default | 2.8%±1.4 | 30 Aug 2026 | Epoch AI log |
| 296 | meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AI | open | default | 2.6%±0.9 | 25 Feb 2025 | Epoch AI log |
| 297 | claude-2.0Anthropic | closed | default | 2.5%±1.5 | 7 Mar 2025 | Epoch AI log |
| 298 | claude-3-sonnet-20240229Anthropic | closed | default | 2.5%±2.0 | 25 Feb 2025 | Epoch AI log |
| 299 | Hermes-2-Theta-Llama-3-70BNous Research,Arcee AIno page match | open | default | 2.5%±1.2 | 1 Apr 2025 | Epoch AI log |
| 300 | Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibaba | open | default | 2.5%±0.9 | 30 Aug 2026 | Epoch AI log |
| 301 | gpt-3.5-turbo-0125OpenAI | closed | default | 2.2%±2.2 | 7 Aug 2026 | Epoch AI log |
| 302 | claude-2.1Anthropic | closed | default | 1.9%±1.0 | 7 Mar 2025 | Epoch AI log |
| 303 | meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AI | open | default | 1.9%±1.0 | 30 Aug 2026 | Epoch AI log |
| 304 | mistral-large-2402Mistral AI | closed | default | 1.9%±0.9 | 25 Feb 2025 | Epoch AI log |
| 305 | claude-3-haiku-20240307Anthropic | closed | default | 1.8%±0.8 | 25 Feb 2025 | Epoch AI log |
| 306 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | default | 1.7%±0.9 | 27 Aug 2026 | Epoch AI log |
| 307 | google/gemma-2-27b-itEpoch: gemma-2-27b-it, Google DeepMind | open | default | 1.4%±0.9 | 7 Mar 2025 | Epoch AI log |
| 308 | gemini-1.0-pro-001Google DeepMind | closed | default | 1.1%±0.6 | 25 Feb 2025 | Epoch AI log |
| 309 | google/gemma-3-1b-itEpoch: gemma-3-1b-it, Google DeepMind | open | default | 1.1%±0.5 | 28 Aug 2026 | Epoch AI log |
| 310 | gpt-4-0613OpenAI | closed | default | 1.1%±0.5 | 23 Oct 2025 | Epoch AI log |
| 311 | deepseek-ai/deepseek-llm-67b-chatEpoch: deepseek-llm-67b-chat, DeepSeek | open | default | 0.8%±0.5 | 28 Aug 2026 | Epoch AI log |
| 312 | ibm-granite/granite-4.0-1bEpoch: granite-4.0-1b | open | default | 0.8%±0.5 | 30 Aug 2026 | Epoch AI log |
| 313 | google/gemma-2-9b-itEpoch: gemma-2-9b-it, Google DeepMind | open | default | 0.6%±0.4 | 7 Mar 2025 | Epoch AI log |
| 314 | gpt-4-0314OpenAI | closed | default | 0.6%±0.4 | 22 Oct 2025 | Epoch AI log |
| 315 | meta-llama/Llama-3.2-1B-InstructEpoch: Llama-3.2-1B-Instruct, Meta AI | open | default | 0.6%±0.4 | 30 Aug 2026 | Epoch AI log |
| 316 | ibm-granite/granite-4.0-350mEpoch: granite-4.0-350m | open | default | 0.3%±0.3 | 28 Aug 2026 | Epoch AI log |
| 317 | mistralai/Mistral-7B-Instruct-v0.3Epoch: Mistral-7B-Instruct-v0.3, Mistral AI | open | default | 0.3%±0.3 | 30 Aug 2026 | Epoch AI log |
| 318 | meta-llama/Llama-2-70b-chat-hfEpoch: Llama-2-70b-chat-hf, Meta AI | open | default | 0.0% | 25 Feb 2025 | Epoch AI log |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method