DTBench
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://conceptualreasoning.ai/dtbench
- Score shown
- Epoch's column "Accuracy", as a percentage, higher is better; random guessing scores about 40.0%. The source gives no error bars.
- Results
- 241 in all: 75 for open-weight models, 161 for closed ones, 5 where Epoch does not say. 59 linked to a model page here.
Best open-weight result: deepseek-ai/DeepSeek-V4-Pro-0813, 93.9%, number 66 of 241 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-fable-5-2_unknownAnthropic | closed | effort unknown | 99.2% | not dated | conceptualreasoning.ai |
| 2 | claude-fable-5-2_xhighAnthropic | closed | effort xhigh | 98.9% | not dated | conceptualreasoning.ai |
| 3 | claude-opus-5-5_maxAnthropic | closed | effort max | 98.9% | not dated | conceptualreasoning.ai |
| 4 | claude-fable-5_maxAnthropic | closed | effort max | 98.4% | not dated | conceptualreasoning.ai |
| 5 | claude-fable-5-2_highAnthropic | closed | effort high | 98.1% | not dated | conceptualreasoning.ai |
| 6 | claude-fable-5-2_lowAnthropic | closed | effort low | 98.1% | not dated | conceptualreasoning.ai |
| 7 | claude-fable-5-2_mediumAnthropic | closed | effort medium | 98.1% | not dated | conceptualreasoning.ai |
| 8 | claude-opus-5_highAnthropic | closed | effort high | 97.9% | not dated | conceptualreasoning.ai |
| 9 | claude-opus-5_xhighAnthropic | closed | effort xhigh | 97.9% | not dated | conceptualreasoning.ai |
| 10 | claude-fable-5-1_unknownAnthropic | closed | effort unknown | 97.6% | not dated | conceptualreasoning.ai |
| 11 | claude-fable-5-1_xhighAnthropic | closed | effort xhigh | 97.6% | not dated | conceptualreasoning.ai |
| 12 | claude-opus-5_maxAnthropic | closed | effort max | 97.6% | not dated | conceptualreasoning.ai |
| 13 | gemini-4-argon_lowGoogle DeepMind | closed | effort low | 97.6% | not dated | conceptualreasoning.ai |
| 14 | claude-fable-5_xhighAnthropic | closed | effort xhigh | 97.3% | not dated | conceptualreasoning.ai |
| 15 | claude-opus-5-5_xhighAnthropic | closed | effort xhigh | 97.3% | not dated | conceptualreasoning.ai |
| 16 | claude-sonnet-5-5_maxAnthropic | closed | effort max | 97.3% | not dated | conceptualreasoning.ai |
| 17 | gemini-4-argon_highGoogle DeepMind | closed | effort high | 97.3% | not dated | conceptualreasoning.ai |
| 18 | gemini-4-argon_mediumGoogle DeepMind | closed | effort medium | 97.3% | not dated | conceptualreasoning.ai |
| 19 | gpt-6-astra_xhighOpenAI | closed | effort xhigh | 97.3% | not dated | conceptualreasoning.ai |
| 20 | gpt-6-sol_maxOpenAI | closed | effort max | 97.3% | not dated | conceptualreasoning.ai |
| 21 | grok-4.6_xhighxAI | closed | effort xhigh | 97.3% | not dated | conceptualreasoning.ai |
| 22 | claude-fable-5-1_mediumAnthropic | closed | effort medium | 97.1% | not dated | conceptualreasoning.ai |
| 23 | gemini-3.1-pro-preview_mediumGoogle DeepMind | closed | effort medium | 97.1% | not dated | conceptualreasoning.ai |
| 24 | gpt-6-astra_mediumOpenAI | closed | effort medium | 97.1% | not dated | conceptualreasoning.ai |
| 25 | gpt-6-astra_unknownOpenAI | closed | effort unknown | 97.1% | not dated | conceptualreasoning.ai |
| 26 | gemini-3.7-flash_highGoogle DeepMind | closed | effort high | 96.8% | not dated | conceptualreasoning.ai |
| 27 | claude-fable-5_highAnthropic | closed | effort high | 96.5% | not dated | conceptualreasoning.ai |
| 28 | claude-fable-5-1_highAnthropic | closed | effort high | 96.5% | not dated | conceptualreasoning.ai |
| 29 | claude-opus-5-5_highAnthropic | closed | effort high | 96.5% | not dated | conceptualreasoning.ai |
| 30 | claude-opus-5-5_mediumAnthropic | closed | effort medium | 96.5% | not dated | conceptualreasoning.ai |
| 31 | gpt-6-astra_highOpenAI | closed | effort high | 96.5% | not dated | conceptualreasoning.ai |
| 32 | gpt-6.1-sol_maxOpenAI | closed | effort max | 96.5% | not dated | conceptualreasoning.ai |
| 33 | grok-4.5_highxAI | closed | effort high | 96.5% | not dated | conceptualreasoning.ai |
| 34 | muse-spark-1.3_unknownMeta AI | closed | effort unknown | 96.5% | not dated | conceptualreasoning.ai |
| 35 | claude-sonnet-5-5_xhighAnthropic | closed | effort xhigh | 96.0% | not dated | conceptualreasoning.ai |
| 36 | gpt-5.5_xhighOpenAI | closed | effort xhigh | 96.0% | not dated | conceptualreasoning.ai |
| 37 | gpt-5.5-pro_xhighOpenAI | closed | effort xhigh | 96.0% | not dated | conceptualreasoning.ai |
| 38 | gpt-5.6-sol_promaxOpenAI | closed | default | 96.0% | not dated | conceptualreasoning.ai |
| 39 | grok-4.7_xhighxAI | closed | effort xhigh | 96.0% | not dated | conceptualreasoning.ai |
| 40 | claude-fable-5_mediumAnthropic | closed | effort medium | 95.7% | not dated | conceptualreasoning.ai |
| 41 | gpt-5.6-sol_xhighOpenAI | closed | effort xhigh | 95.7% | not dated | conceptualreasoning.ai |
| 42 | gemini-3.8-flash_unknownGoogle DeepMind | closed | effort unknown | 95.7% | not dated | conceptualreasoning.ai |
| 43 | claude-opus-5-5_lowAnthropic | closed | effort low | 95.5% | not dated | conceptualreasoning.ai |
| 44 | gemini-3.1-pro-preview_highGoogle DeepMind | closed | effort high | 95.5% | not dated | conceptualreasoning.ai |
| 45 | gemini-3.6-flash_highGoogle DeepMind | closed | effort high | 95.5% | not dated | conceptualreasoning.ai |
| 46 | gpt-5.6-sol_highOpenAI | closed | effort high | 95.5% | not dated | conceptualreasoning.ai |
| 47 | gpt-5.6-sol_maxOpenAI | closed | effort max | 95.5% | not dated | conceptualreasoning.ai |
| 48 | gpt-5.6-sol_mediumOpenAI | closed | effort medium | 95.2% | not dated | conceptualreasoning.ai |
| 49 | muse-wonder-1.0_unknownMeta AI | closed | effort unknown | 95.2% | not dated | conceptualreasoning.ai |
| 50 | claude-haiku-5-5_maxAnthropic | closed | effort max | 95.0% | not dated | conceptualreasoning.ai |
| 51 | claude-opus-4-8_maxAnthropic | closed | effort max | 94.9% | not dated | conceptualreasoning.ai |
| 52 | claude-opus-5_mediumAnthropic | closed | effort medium | 94.9% | not dated | conceptualreasoning.ai |
| 53 | gemini-3.1-pro-preview_lowGoogle DeepMind | closed | effort low | 94.9% | not dated | conceptualreasoning.ai |
| 54 | gemini-3.8-flash_lowGoogle DeepMind | closed | effort low | 94.9% | not dated | conceptualreasoning.ai |
| 55 | gemini-3.8-flash_mediumGoogle DeepMind | closed | effort medium | 94.9% | not dated | conceptualreasoning.ai |
| 56 | gpt-6-astra_lowOpenAI | closed | effort low | 94.9% | not dated | conceptualreasoning.ai |
| 57 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 94.7% | not dated | conceptualreasoning.ai |
| 58 | gemini-3.7-flash_mediumGoogle DeepMind | closed | effort medium | 94.7% | not dated | conceptualreasoning.ai |
| 59 | muse-spark-1.2_xhighMeta AI | closed | effort xhigh | 94.7% | not dated | conceptualreasoning.ai |
| 60 | claude-opus-4-7_maxAnthropic | closed | effort max | 94.7% | not dated | conceptualreasoning.ai |
| 61 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 94.4% | not dated | conceptualreasoning.ai |
| 62 | muse-spark-1.1_highMeta AI | closed | effort high | 94.4% | not dated | conceptualreasoning.ai |
| 63 | claude-fable-5-1_lowAnthropic | closed | effort low | 94.1% | not dated | conceptualreasoning.ai |
| 64 | claude-sonnet-5-5_highAnthropic | closed | effort high | 94.1% | not dated | conceptualreasoning.ai |
| 65 | claude-sonnet-5-5_mediumAnthropic | closed | effort medium | 93.9% | not dated | conceptualreasoning.ai |
| 66 | deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_unknown, DeepSeek | open | effort unknown | 93.9% | not dated | conceptualreasoning.ai |
| 67 | gpt-5.6-sol_lowOpenAI | closed | effort low | 93.9% | not dated | conceptualreasoning.ai |
| 68 | zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI) | open | effort max | 93.6% | not dated | conceptualreasoning.ai |
| 69 | claude-sonnet-5-5_lowAnthropic | closed | effort low | 93.3% | not dated | conceptualreasoning.ai |
| 70 | gemini-3.7-flash_lowGoogle DeepMind | closed | effort low | 93.3% | not dated | conceptualreasoning.ai |
| 71 | gpt-5.6-terra_maxOpenAI | closed | effort max | 93.3% | not dated | conceptualreasoning.ai |
| 72 | moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro_max, Moonshot | open | effort max | 93.3% | not dated | conceptualreasoning.ai |
| 73 | claude-opus-5_lowAnthropic | closed | effort low | 93.1% | not dated | conceptualreasoning.ai |
| 74 | gpt-5.6-terra_xhighOpenAI | closed | effort xhigh | 92.5% | not dated | conceptualreasoning.ai |
| 75 | claude-sonnet-5_maxAnthropic | closed | effort max | 92.5% | not dated | conceptualreasoning.ai |
| 76 | qwen3.7-maxAlibaba | closed | default | 92.3% | not dated | conceptualreasoning.ai |
| 77 | qwen3.8-max_xhighAlibaba | closed | effort xhigh | 92.0% | not dated | conceptualreasoning.ai |
| 78 | claude-fable-5_lowAnthropic | closed | effort low | 91.5% | not dated | conceptualreasoning.ai |
| 79 | claude-opus-4-6_maxAnthropic | closed | effort max | 91.2% | not dated | conceptualreasoning.ai |
| 80 | gpt-5.6-terra_highOpenAI | closed | effort high | 91.2% | not dated | conceptualreasoning.ai |
| 81 | moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshot | open | effort max | 91.2% | not dated | conceptualreasoning.ai |
| 82 | deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeek | open | effort max | 90.9% | not dated | conceptualreasoning.ai |
| 83 | gpt-5.2-2025-12-11_xhighOpenAI | closed | effort xhigh | 90.9% | not dated | conceptualreasoning.ai |
| 84 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 90.9% | not dated | conceptualreasoning.ai |
| 85 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeek | open | effort max | 90.7% | not dated | conceptualreasoning.ai |
| 86 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 90.7% | not dated | conceptualreasoning.ai |
| 87 | grok-4.3_highxAI | closed | effort high | 90.7% | not dated | conceptualreasoning.ai |
| 88 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 90.1% | not dated | conceptualreasoning.ai |
| 89 | gpt-5.6-terra_lowOpenAI | closed | effort low | 90.1% | not dated | conceptualreasoning.ai |
| 90 | gpt-5.6-terra_mediumOpenAI | closed | effort medium | 90.1% | not dated | conceptualreasoning.ai |
| 91 | gpt-6-luna_maxOpenAI | closed | effort max | 90.1% | not dated | conceptualreasoning.ai |
| 92 | grok-4-20xAI | closed | default | 90.1% | not dated | conceptualreasoning.ai |
| 93 | nemotron-3-ultraNvidiano page match | open | default | 90.1% | not dated | conceptualreasoning.ai |
| 94 | claude-opus-4-5-20251101_highAnthropic | closed | effort high | 89.9% | not dated | conceptualreasoning.ai |
| 95 | claude-sonnet-4-6_maxAnthropic | closed | effort max | 89.9% | not dated | conceptualreasoning.ai |
| 96 | deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_unknown, DeepSeek | open | effort unknown | 89.9% | not dated | conceptualreasoning.ai |
| 97 | claude-sonnet-5_xhighAnthropic | closed | effort xhigh | 89.5% | not dated | conceptualreasoning.ai |
| 98 | gemini-3-flash-preview_highGoogle DeepMind | closed | effort high | 89.1% | not dated | conceptualreasoning.ai |
| 99 | gpt-5.6-luna_xhighOpenAI | closed | effort xhigh | 89.1% | not dated | conceptualreasoning.ai |
| 100 | gpt-5.6-luna_maxOpenAI | closed | effort max | 88.8% | not dated | conceptualreasoning.ai |
| 101 | mistral-large-4-0_unknownMistral AIno page match | open | effort unknown | 88.2% | not dated | conceptualreasoning.ai |
| 102 | Qwen/Qwen3.8-27BEpoch: qwen3.8-27b, Alibaba | open | default | 88.0% | not dated | conceptualreasoning.ai |
| 103 | deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_thinking, DeepSeek | open | effort thinking | 87.7% | not dated | conceptualreasoning.ai |
| 104 | grok-4-1-fast-reasoningxAI | closed | default | 87.7% | not dated | conceptualreasoning.ai |
| 105 | zai-org/GLM-5.3Epoch: glm-5.3_unknown, Z.ai (Zhipu AI) | open | effort unknown | 87.7% | not dated | conceptualreasoning.ai |
| 106 | thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machines | open | effort xhigh | 87.5% | not dated | conceptualreasoning.ai |
| 107 | Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibaba | open | default | 87.5% | not dated | conceptualreasoning.ai |
| 108 | claude-haiku-5-5_xhighAnthropic | closed | effort xhigh | 87.2% | not dated | conceptualreasoning.ai |
| 109 | qwen3.6-max-previewAlibaba | not stated | default | 87.2% | not dated | conceptualreasoning.ai |
| 110 | o3-pro-2025-06-10_highOpenAI | closed | effort high | 86.9% | not dated | conceptualreasoning.ai |
| 111 | claude-haiku-5-5_highAnthropic | closed | effort high | 86.7% | not dated | conceptualreasoning.ai |
| 112 | gpt-5.6-luna_highOpenAI | closed | effort high | 86.7% | not dated | conceptualreasoning.ai |
| 113 | deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_max, DeepSeek | open | effort max | 86.4% | not dated | conceptualreasoning.ai |
| 114 | deepseek-ai/DeepSeek-V3.2Epoch: deepseek/deepseek-v3.2, DeepSeek | open | default | 85.6% | not dated | conceptualreasoning.ai |
| 115 | o3-2025-04-16_highOpenAI | closed | effort high | 84.8% | not dated | conceptualreasoning.ai |
| 116 | claude-sonnet-5_highAnthropic | closed | effort high | 84.5% | not dated | conceptualreasoning.ai |
| 117 | XiaomiMiMo/MiMo-V2.5-ProEpoch: mimo-v2.5-pro, Xiaomi Corp | open | default | 84.5% | not dated | conceptualreasoning.ai |
| 118 | claude-haiku-5-5_mediumAnthropic | closed | effort medium | 84.3% | not dated | conceptualreasoning.ai |
| 119 | Qwen/Qwen3.5-122B-A10BEpoch: qwen3.5-122b-a10b_none, Alibaba | open | effort none | 84.3% | not dated | conceptualreasoning.ai |
| 120 | qwen3.7-plusAlibaba | closed | default | 84.0% | not dated | conceptualreasoning.ai |
| 121 | gpt-5.6-sol_noneOpenAI | closed | effort none | 83.7% | not dated | conceptualreasoning.ai |
| 122 | gemini-3.5-flash-lite_highGoogle DeepMind | closed | effort high | 83.5% | not dated | conceptualreasoning.ai |
| 123 | claude-sonnet-4-5-20250929_unknownAnthropic | closed | effort unknown | 83.2% | not dated | conceptualreasoning.ai |
| 124 | gpt-5.6-luna_mediumOpenAI | closed | effort medium | 83.2% | not dated | conceptualreasoning.ai |
| 125 | qwen3.5-flashAlibaba | not stated | default | 82.9% | not dated | conceptualreasoning.ai |
| 126 | deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1_thinking, DeepSeek | open | effort thinking | 82.7% | not dated | conceptualreasoning.ai |
| 127 | google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMind | open | default | 82.7% | not dated | conceptualreasoning.ai |
| 128 | grok-4-fastxAI | closed | default | 82.7% | not dated | conceptualreasoning.ai |
| 129 | gemini-2.5-proGoogle DeepMind | closed | default | 82.4% | not dated | conceptualreasoning.ai |
| 130 | Qwen/Qwen3.5-27BEpoch: qwen3.5-27B, Alibaba | open | default | 82.4% | not dated | conceptualreasoning.ai |
| 131 | qwen3-max-2025-09-23Alibaba | closed | default | 82.1% | not dated | conceptualreasoning.ai |
| 132 | qwen3.6-plusAlibaba | closed | default | 81.9% | not dated | conceptualreasoning.ai |
| 133 | claude-opus-4-20250514_unknownAnthropic | closed | effort unknown | 81.6% | not dated | conceptualreasoning.ai |
| 134 | deepseek-ai/DeepSeek-V3.1-TerminusEpoch: DeepSeek-V3.1-Terminus, DeepSeek | open | default | 81.3% | not dated | conceptualreasoning.ai |
| 135 | claude-haiku-5-5_lowAnthropic | closed | effort low | 81.1% | not dated | conceptualreasoning.ai |
| 136 | qwen-plus-2025-04-28Alibaba | closed | default | 81.1% | not dated | conceptualreasoning.ai |
| 137 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 80.5% | not dated | conceptualreasoning.ai |
| 138 | qwen3.5-plusAlibaba | not stated | default | 80.5% | not dated | conceptualreasoning.ai |
| 139 | gpt-5.4-nano-2026-03-17_xhighOpenAI | closed | effort xhigh | 80.3% | not dated | conceptualreasoning.ai |
| 140 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 80.3% | not dated | conceptualreasoning.ai |
| 141 | claude-opus-4-1-20250805_unknownAnthropic | closed | effort unknown | 80.0% | not dated | conceptualreasoning.ai |
| 142 | gpt-5.4-mini-2026-03-17_xhighOpenAI | closed | effort xhigh | 80.0% | not dated | conceptualreasoning.ai |
| 143 | gpt-5.6-luna_lowOpenAI | closed | effort low | 80.0% | not dated | conceptualreasoning.ai |
| 144 | Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibaba | open | default | 80.0% | not dated | conceptualreasoning.ai |
| 145 | claude-sonnet-5_mediumAnthropic | closed | effort medium | 79.2% | not dated | conceptualreasoning.ai |
| 146 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMax | open | default | 78.9% | not dated | conceptualreasoning.ai |
| 147 | gpt-5.6-terra_noneOpenAI | closed | effort none | 78.7% | not dated | conceptualreasoning.ai |
| 148 | Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: Qwen3-235B-A22B-Instruct-2507, Alibaba | open | default | 78.4% | not dated | conceptualreasoning.ai |
| 149 | Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibaba | open | default | 78.1% | not dated | conceptualreasoning.ai |
| 150 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 77.6% | not dated | conceptualreasoning.ai |
| 151 | claude-sonnet-4-20250514_unknownAnthropic | closed | effort unknown | 77.1% | not dated | conceptualreasoning.ai |
| 152 | qwen3.6-flashAlibaba | not stated | default | 77.1% | not dated | conceptualreasoning.ai |
| 153 | gemini-3.1-flash-lite_highGoogle | closed | effort high | 76.8% | not dated | conceptualreasoning.ai |
| 154 | gemini-2.5-flashGoogle DeepMind | closed | default | 76.5% | not dated | conceptualreasoning.ai |
| 155 | openai/gpt-oss-120bEpoch: gpt-oss-120b_high, OpenAI | open | effort high | 76.3% | not dated | conceptualreasoning.ai |
| 156 | Qwen/Qwen3-235B-A22BEpoch: Qwen3-235B-A22B, Alibaba | open | default | 75.7% | not dated | conceptualreasoning.ai |
| 157 | mistral-medium-2604Mistral AIno page match | open | default | 75.5% | not dated | conceptualreasoning.ai |
| 158 | claude-haiku-5-5_noneAnthropic | closed | effort none | 75.2% | not dated | conceptualreasoning.ai |
| 159 | gemma-4-26b-a4bGoogle DeepMindno page match | open | default | 74.9% | not dated | conceptualreasoning.ai |
| 160 | o1-2024-12-17_highOpenAI | closed | effort high | 74.7% | not dated | conceptualreasoning.ai |
| 161 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibaba | open | default | 73.9% | not dated | conceptualreasoning.ai |
| 162 | claude-haiku-4-5-20251001_unknownAnthropic | closed | effort unknown | 73.6% | not dated | conceptualreasoning.ai |
| 163 | claude-sonnet-5_lowAnthropic | closed | effort low | 73.6% | not dated | conceptualreasoning.ai |
| 164 | Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibaba | open | default | 71.2% | not dated | conceptualreasoning.ai |
| 165 | mistral-small-2603Mistral AIno page match | open | default | 70.9% | not dated | conceptualreasoning.ai |
| 166 | Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibaba | open | default | 69.3% | not dated | conceptualreasoning.ai |
| 167 | gpt-5.6-luna_noneOpenAI | closed | effort none | 69.3% | not dated | conceptualreasoning.ai |
| 168 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 68.8% | not dated | conceptualreasoning.ai |
| 169 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 68.8% | not dated | conceptualreasoning.ai |
| 170 | gpt-4.1-2025-04-14OpenAI | closed | default | 68.3% | not dated | conceptualreasoning.ai |
| 171 | openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAI | open | effort high | 68.0% | not dated | conceptualreasoning.ai |
| 172 | claude-3-5-sonnet-20241022Anthropic | closed | default | 67.8% | not dated | conceptualreasoning.ai |
| 173 | claude-3-5-sonnet-20240620Anthropic | closed | default | 67.8% | not dated | conceptualreasoning.ai |
| 174 | mistral-medium-2508Mistral AI | closed | default | 67.5% | not dated | conceptualreasoning.ai |
| 175 | Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibaba | open | default | 67.5% | not dated | conceptualreasoning.ai |
| 176 | Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibaba | open | default | 67.2% | not dated | conceptualreasoning.ai |
| 177 | grok-2-1212xAI | closed | default | 65.2% | not dated | conceptualreasoning.ai |
| 178 | mistral-large-2512Mistral AIno page match | open | default | 65.1% | not dated | conceptualreasoning.ai |
| 179 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 64.8% | not dated | conceptualreasoning.ai |
| 180 | gpt-4o-2024-05-13OpenAI | closed | default | 64.5% | not dated | conceptualreasoning.ai |
| 181 | Qwen/Qwen3-14BEpoch: qwen3-14b, Alibaba | open | default | 64.0% | not dated | conceptualreasoning.ai |
| 182 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 63.2% | not dated | conceptualreasoning.ai |
| 183 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 62.9% | not dated | conceptualreasoning.ai |
| 184 | gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMind | closed | default | 62.8% | not dated | conceptualreasoning.ai |
| 185 | deepseek-chatDeepSeekno page match | open | default | 62.7% | not dated | conceptualreasoning.ai |
| 186 | gpt-4-0613OpenAI | closed | default | 62.7% | not dated | conceptualreasoning.ai |
| 187 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 62.7% | not dated | conceptualreasoning.ai |
| 188 | mistral-medium-2505Mistral AI | closed | default | 62.3% | not dated | conceptualreasoning.ai |
| 189 | meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AI | open | default | 61.9% | not dated | conceptualreasoning.ai |
| 190 | claude-3-opus-20240229Anthropic | closed | default | 61.6% | not dated | conceptualreasoning.ai |
| 191 | gpt-4-turboOpenAI | closed | default | 61.6% | not dated | conceptualreasoning.ai |
| 192 | meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AI | open | default | 61.4% | not dated | conceptualreasoning.ai |
| 193 | CohereLabs/c4ai-command-a-03-2025Epoch: c4ai-command-a-03-2025, Cohere | open | default | 61.3% | not dated | conceptualreasoning.ai |
| 194 | mistralai/Magistral-Small-2509Epoch: magistral-small-2509, Mistral AI | open | default | 61.3% | not dated | conceptualreasoning.ai |
| 195 | mistral-large-2407Mistral AIno page match | open | default | 61.2% | not dated | conceptualreasoning.ai |
| 196 | mistral-large-2411Mistral AIno page match | open | default | 60.8% | not dated | conceptualreasoning.ai |
| 197 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibaba | open | default | 60.3% | not dated | conceptualreasoning.ai |
| 198 | meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AI | open | default | 60.0% | not dated | conceptualreasoning.ai |
| 199 | mistral-small-2506Mistral AIno page match | open | default | 59.9% | not dated | conceptualreasoning.ai |
| 200 | Qwen/Qwen3-8BEpoch: qwen3-8b, Alibaba | open | default | 59.7% | not dated | conceptualreasoning.ai |
| 201 | meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AI | open | default | 59.5% | not dated | conceptualreasoning.ai |
| 202 | gemini-1.5-pro-002Google DeepMind | closed | default | 59.0% | not dated | conceptualreasoning.ai |
| 203 | mistral-small-2503Mistral AIno page match | open | default | 58.6% | not dated | conceptualreasoning.ai |
| 204 | meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AI | open | default | 57.9% | not dated | conceptualreasoning.ai |
| 205 | claude-3-5-haiku-20241022Anthropic | closed | default | 56.7% | not dated | conceptualreasoning.ai |
| 206 | gemini-1.5-pro-001Google DeepMind | closed | default | 56.5% | not dated | conceptualreasoning.ai |
| 207 | mistral-large-2402Mistral AI | closed | default | 55.9% | not dated | conceptualreasoning.ai |
| 208 | mistralai/Mixtral-8x22B-Instruct-v0.1Epoch: Mixtral-8x22B-Instruct-v0.1, Mistral AI | open | default | 55.1% | not dated | conceptualreasoning.ai |
| 209 | CohereLabs/c4ai-command-r-plus-08-2024Epoch: c4ai-command-r-plus-08-2024, Cohere,Cohere Labs (formerly Cohere for AI) | open | default | 54.9% | not dated | conceptualreasoning.ai |
| 210 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 54.4% | not dated | conceptualreasoning.ai |
| 211 | meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AI | open | default | 54.2% | not dated | conceptualreasoning.ai |
| 212 | gemini-1.5-flash-002Google DeepMind | closed | default | 53.8% | not dated | conceptualreasoning.ai |
| 213 | claude-3-sonnet-20240229Anthropic | closed | default | 53.6% | not dated | conceptualreasoning.ai |
| 214 | mistral-medium-2312Mistral AI | closed | default | 53.3% | not dated | conceptualreasoning.ai |
| 215 | mistral-small-2402 | not stated | default | 53.0% | not dated | conceptualreasoning.ai |
| 216 | gemini-2.0-flash-liteGoogle DeepMind | closed | default | 52.5% | not dated | conceptualreasoning.ai |
| 217 | google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMind | open | default | 52.5% | not dated | conceptualreasoning.ai |
| 218 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 52.5% | not dated | conceptualreasoning.ai |
| 219 | claude-2.0Anthropic | closed | default | 51.8% | not dated | conceptualreasoning.ai |
| 220 | ministral-3b-2410Mistral AI | closed | default | 51.7% | not dated | conceptualreasoning.ai |
| 221 | gemini-1.5-flash-001Google DeepMind | closed | default | 51.4% | not dated | conceptualreasoning.ai |
| 222 | claude-2.1Anthropic | closed | default | 50.9% | not dated | conceptualreasoning.ai |
| 223 | google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMind | open | default | 50.9% | not dated | conceptualreasoning.ai |
| 224 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | default | 50.9% | not dated | conceptualreasoning.ai |
| 225 | claude-3-haiku-20240307Anthropic | closed | default | 50.1% | not dated | conceptualreasoning.ai |
| 226 | gemini-1.5-flash-8b-001Google DeepMind | closed | default | 50.0% | not dated | conceptualreasoning.ai |
| 227 | open-mixtral-8x7bMistral AIno page match | open | default | 49.6% | not dated | conceptualreasoning.ai |
| 228 | mistral-small-2409Mistral AIno page match | open | default | 49.5% | not dated | conceptualreasoning.ai |
| 229 | google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMind | open | default | 48.8% | not dated | conceptualreasoning.ai |
| 230 | mistralai/Mistral-Nemo-Instruct-2407Epoch: Mistral-Nemo-Instruct-2407, Mistral AI | open | default | 48.6% | not dated | conceptualreasoning.ai |
| 231 | gpt-3.5-turbo-0125OpenAI | closed | default | 48.5% | not dated | conceptualreasoning.ai |
| 232 | gemma-2-27bGoogle DeepMindno page match | open | default | 48.0% | not dated | conceptualreasoning.ai |
| 233 | Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibaba | open | default | 47.7% | not dated | conceptualreasoning.ai |
| 234 | CohereLabs/c4ai-command-r-08-2024Epoch: c4ai-command-r-08-2024 | open | default | 46.4% | not dated | conceptualreasoning.ai |
| 235 | gemini-1.0-pro-001Google DeepMind | closed | default | 45.9% | not dated | conceptualreasoning.ai |
| 236 | claude-instant-1.2Anthropic | closed | default | 45.8% | not dated | conceptualreasoning.ai |
| 237 | ministral-8b-2410Mistral AIno page match | open | default | 45.7% | not dated | conceptualreasoning.ai |
| 238 | meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AI | open | default | 43.9% | not dated | conceptualreasoning.ai |
| 239 | open-mistral-7bMistral AIno page match | open | default | 42.5% | not dated | conceptualreasoning.ai |
| 240 | meta-llama/Llama-2-13b-chatEpoch: Llama-2-13b-chat, Meta AI | open | default | 42.2% | not dated | conceptualreasoning.ai |
| 241 | meta-llama/Llama-2-70b-chatEpoch: Llama-2-70b-chat, Meta AI | open | default | 41.6% | not dated | conceptualreasoning.ai |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method