LMCA
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://conceptualreasoning.ai/lmca
- Score shown
- Epoch's column "Score", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
- Results
- 203 in all: 59 for open-weight models, 140 for closed ones, 4 where Epoch does not say. 51 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K3.1-Pro, 57.6%, number 44 of 203 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-fable-5-2_xhighAnthropic | closed | effort xhigh | 70.6%68.0 to 73.2 | not dated | conceptualreasoning.ai |
| 2 | claude-fable-5-2_unknownAnthropic | closed | effort unknown | 70.1%67.5 to 72.7 | not dated | conceptualreasoning.ai |
| 3 | claude-fable-5-2_mediumAnthropic | closed | effort medium | 70.0%67.4 to 72.6 | not dated | conceptualreasoning.ai |
| 4 | claude-fable-5-2_highAnthropic | closed | effort high | 69.9%67.3 to 72.5 | not dated | conceptualreasoning.ai |
| 5 | claude-fable-5-2_lowAnthropic | closed | effort low | 69.6%67.0 to 72.3 | not dated | conceptualreasoning.ai |
| 6 | claude-sonnet-5-5_maxAnthropic | closed | effort max | 68.3%65.5 to 71.0 | not dated | conceptualreasoning.ai |
| 7 | claude-opus-5-5_maxAnthropic | closed | effort max | 68.2%65.5 to 71.0 | not dated | conceptualreasoning.ai |
| 8 | claude-sonnet-5-5_xhighAnthropic | closed | effort xhigh | 66.3%63.4 to 69.2 | not dated | conceptualreasoning.ai |
| 9 | claude-opus-5-5_xhighAnthropic | closed | effort xhigh | 66.2%63.3 to 69.1 | not dated | conceptualreasoning.ai |
| 10 | claude-sonnet-5-5_highAnthropic | closed | effort high | 65.9%63.0 to 68.8 | not dated | conceptualreasoning.ai |
| 11 | claude-opus-5-5_highAnthropic | closed | effort high | 65.9%63.0 to 68.8 | not dated | conceptualreasoning.ai |
| 12 | claude-opus-5-5_mediumAnthropic | closed | effort medium | 65.7%62.7 to 68.6 | not dated | conceptualreasoning.ai |
| 13 | claude-fable-5-1_unknownAnthropic | closed | effort unknown | 65.5%62.5 to 68.4 | not dated | conceptualreasoning.ai |
| 14 | claude-sonnet-5-5_mediumAnthropic | closed | effort medium | 65.5%62.5 to 68.4 | not dated | conceptualreasoning.ai |
| 15 | claude-fable-5-1_mediumAnthropic | closed | effort medium | 65.3%62.4 to 68.3 | not dated | conceptualreasoning.ai |
| 16 | claude-fable-5-1_highAnthropic | closed | effort high | 65.1%62.1 to 68.0 | not dated | conceptualreasoning.ai |
| 17 | claude-fable-5-1_xhighAnthropic | closed | effort xhigh | 65.1%62.1 to 68.0 | not dated | conceptualreasoning.ai |
| 18 | claude-opus-5-5_lowAnthropic | closed | effort low | 64.9%62.0 to 67.9 | not dated | conceptualreasoning.ai |
| 19 | claude-sonnet-5-5_lowAnthropic | closed | effort low | 64.5%61.5 to 67.5 | not dated | conceptualreasoning.ai |
| 20 | claude-opus-5_xhighAnthropic | closed | effort xhigh | 64.5%61.5 to 67.5 | not dated | conceptualreasoning.ai |
| 21 | gpt-6-astra_xhighOpenAI | closed | effort xhigh | 64.4%61.4 to 67.4 | not dated | conceptualreasoning.ai |
| 22 | gpt-6-astra_unknownOpenAI | closed | effort unknown | 64.1%61.1 to 67.1 | not dated | conceptualreasoning.ai |
| 23 | gpt-6-astra_highOpenAI | closed | effort high | 63.8%60.8 to 66.9 | not dated | conceptualreasoning.ai |
| 24 | gemini-4-argon_highGoogle DeepMind | closed | effort high | 63.7%60.6 to 66.7 | not dated | conceptualreasoning.ai |
| 25 | claude-opus-5_highAnthropic | closed | effort high | 63.7%60.6 to 66.7 | not dated | conceptualreasoning.ai |
| 26 | claude-fable-5-1_lowAnthropic | closed | effort low | 63.5%60.4 to 66.5 | not dated | conceptualreasoning.ai |
| 27 | claude-opus-5_maxAnthropic | closed | effort max | 63.3%60.2 to 66.4 | not dated | conceptualreasoning.ai |
| 28 | claude-opus-5_mediumAnthropic | closed | effort medium | 62.9%59.8 to 66.0 | not dated | conceptualreasoning.ai |
| 29 | gpt-6-astra_mediumOpenAI | closed | effort medium | 62.9%59.8 to 66.0 | not dated | conceptualreasoning.ai |
| 30 | gemini-4-argon_lowGoogle DeepMind | closed | effort low | 62.9%59.8 to 66.0 | not dated | conceptualreasoning.ai |
| 31 | gemini-4-argon_mediumGoogle DeepMind | closed | effort medium | 62.8%59.7 to 66.0 | not dated | conceptualreasoning.ai |
| 32 | gpt-6.1-sol_maxOpenAI | closed | effort max | 62.8%59.7 to 65.9 | not dated | conceptualreasoning.ai |
| 33 | claude-opus-5_lowAnthropic | closed | effort low | 62.6%59.5 to 65.7 | not dated | conceptualreasoning.ai |
| 34 | gpt-6-astra_lowOpenAI | closed | effort low | 62.2%59.0 to 65.3 | not dated | conceptualreasoning.ai |
| 35 | claude-fable-5_highAnthropic | closed | effort high | 61.2%57.9 to 64.4 | not dated | conceptualreasoning.ai |
| 36 | claude-fable-5_xhighAnthropic | closed | effort xhigh | 61.0%57.8 to 64.2 | not dated | conceptualreasoning.ai |
| 37 | claude-fable-5_maxAnthropic | closed | effort max | 60.3%57.1 to 63.6 | not dated | conceptualreasoning.ai |
| 38 | claude-fable-5_mediumAnthropic | closed | effort medium | 60.3%57.1 to 63.6 | not dated | conceptualreasoning.ai |
| 39 | claude-fable-5_lowAnthropic | closed | effort low | 59.6%56.3 to 62.9 | not dated | conceptualreasoning.ai |
| 40 | gpt-5.6-sol_promaxOpenAI | closed | default | 59.2%55.9 to 62.5 | not dated | conceptualreasoning.ai |
| 41 | gpt-6-sol_maxOpenAI | closed | effort max | 59.1%55.7 to 62.4 | not dated | conceptualreasoning.ai |
| 42 | gpt-5.6-sol_xhighOpenAI | closed | effort xhigh | 58.5%55.1 to 61.9 | not dated | conceptualreasoning.ai |
| 43 | gpt-5.6-sol_maxOpenAI | closed | effort max | 58.4%55.0 to 61.8 | not dated | conceptualreasoning.ai |
| 44 | moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro_max, Moonshot | open | effort max | 57.6%54.2 to 61.0 | not dated | conceptualreasoning.ai |
| 45 | claude-opus-4-8_maxAnthropic | closed | effort max | 57.5%54.1 to 60.9 | not dated | conceptualreasoning.ai |
| 46 | gpt-5.6-sol_highOpenAI | closed | effort high | 56.9%53.5 to 60.4 | not dated | conceptualreasoning.ai |
| 47 | muse-wonder-1.0_unknownMeta AI | closed | effort unknown | 56.8%53.4 to 60.3 | not dated | conceptualreasoning.ai |
| 48 | gpt-5.6-sol_mediumOpenAI | closed | effort medium | 56.4%52.9 to 59.9 | not dated | conceptualreasoning.ai |
| 49 | gpt-5.6-sol_lowOpenAI | closed | effort low | 55.9%52.4 to 59.4 | not dated | conceptualreasoning.ai |
| 50 | claude-opus-4-6_maxAnthropic | closed | effort max | 55.8%52.2 to 59.3 | not dated | conceptualreasoning.ai |
| 51 | zai-org/GLM-5.3Epoch: glm-5.3_unknown, Z.ai (Zhipu AI) | open | effort unknown | 55.5%52.0 to 59.1 | not dated | conceptualreasoning.ai |
| 52 | gpt-5.6-terra_maxOpenAI | closed | effort max | 54.9%51.4 to 58.5 | not dated | conceptualreasoning.ai |
| 53 | gpt-5.5_xhighOpenAI | closed | effort xhigh | 54.3%50.6 to 57.9 | not dated | conceptualreasoning.ai |
| 54 | gpt-5.5-pro_xhighOpenAI | closed | effort xhigh | 53.9%50.3 to 57.6 | not dated | conceptualreasoning.ai |
| 55 | muse-spark-1.3_unknownMeta AI | closed | effort unknown | 53.9%50.2 to 57.5 | not dated | conceptualreasoning.ai |
| 56 | gemini-3.1-pro-preview_highGoogle DeepMind | closed | effort high | 53.8%50.2 to 57.5 | not dated | conceptualreasoning.ai |
| 57 | gemini-3.8-flash_mediumGoogle DeepMind | closed | effort medium | 52.9%49.2 to 56.6 | not dated | conceptualreasoning.ai |
| 58 | gemini-3.1-pro-preview_mediumGoogle DeepMind | closed | effort medium | 52.8%49.1 to 56.5 | not dated | conceptualreasoning.ai |
| 59 | gpt-5.6-sol_noneOpenAI | closed | effort none | 52.8%49.0 to 56.5 | not dated | conceptualreasoning.ai |
| 60 | moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshot | open | effort max | 52.7%49.0 to 56.4 | not dated | conceptualreasoning.ai |
| 61 | gpt-5.6-terra_xhighOpenAI | closed | effort xhigh | 52.2%48.5 to 56.0 | not dated | conceptualreasoning.ai |
| 62 | claude-opus-4-7_maxAnthropic | closed | effort max | 52.2%48.5 to 55.9 | not dated | conceptualreasoning.ai |
| 63 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 52.0%48.2 to 55.7 | not dated | conceptualreasoning.ai |
| 64 | gemini-3.8-flash_unknownGoogle DeepMind | closed | effort unknown | 51.5%47.8 to 55.3 | not dated | conceptualreasoning.ai |
| 65 | gpt-5.6-terra_highOpenAI | closed | effort high | 51.4%47.6 to 55.1 | not dated | conceptualreasoning.ai |
| 66 | gemini-3.1-pro-preview_lowGoogle DeepMind | closed | effort low | 51.2%47.4 to 55.0 | not dated | conceptualreasoning.ai |
| 67 | gemini-3.8-flash_lowGoogle DeepMind | closed | effort low | 51.2%47.4 to 54.9 | not dated | conceptualreasoning.ai |
| 68 | claude-haiku-5-5_maxAnthropic | closed | effort max | 51.1%47.3 to 54.9 | not dated | conceptualreasoning.ai |
| 69 | gemini-3.7-flash_highGoogle DeepMind | closed | effort high | 50.4%46.6 to 54.2 | not dated | conceptualreasoning.ai |
| 70 | claude-sonnet-5_highAnthropic | closed | effort high | 50.0%46.2 to 53.9 | not dated | conceptualreasoning.ai |
| 71 | muse-spark-1.1_highMeta AI | closed | effort high | 49.9%46.1 to 53.8 | not dated | conceptualreasoning.ai |
| 72 | claude-sonnet-5_mediumAnthropic | closed | effort medium | 49.7%45.8 to 53.6 | not dated | conceptualreasoning.ai |
| 73 | claude-sonnet-5_lowAnthropic | closed | effort low | 49.6%45.8 to 53.5 | not dated | conceptualreasoning.ai |
| 74 | gpt-5.6-terra_mediumOpenAI | closed | effort medium | 49.6%45.8 to 53.5 | not dated | conceptualreasoning.ai |
| 75 | grok-4.7_xhighxAI | closed | effort xhigh | 49.4%45.5 to 53.3 | not dated | conceptualreasoning.ai |
| 76 | claude-sonnet-5_maxAnthropic | closed | effort max | 49.3%45.4 to 53.2 | not dated | conceptualreasoning.ai |
| 77 | gpt-5.6-terra_lowOpenAI | closed | effort low | 49.3%45.4 to 53.2 | not dated | conceptualreasoning.ai |
| 78 | gemini-3.7-flash_mediumGoogle DeepMind | closed | effort medium | 49.2%45.3 to 53.1 | not dated | conceptualreasoning.ai |
| 79 | claude-sonnet-5_xhighAnthropic | closed | effort xhigh | 48.8%44.9 to 52.7 | not dated | conceptualreasoning.ai |
| 80 | gpt-5.6-luna_maxOpenAI | closed | effort max | 48.5%44.6 to 52.5 | not dated | conceptualreasoning.ai |
| 81 | gpt-5.6-luna_xhighOpenAI | closed | effort xhigh | 48.5%44.6 to 52.5 | not dated | conceptualreasoning.ai |
| 82 | grok-4.6_xhighxAI | closed | effort xhigh | 48.4%44.5 to 52.4 | not dated | conceptualreasoning.ai |
| 83 | muse-spark-1.2_xhighMeta AI | closed | effort xhigh | 48.4%44.4 to 52.3 | not dated | conceptualreasoning.ai |
| 84 | claude-haiku-5-5_xhighAnthropic | closed | effort xhigh | 48.0%44.0 to 51.9 | not dated | conceptualreasoning.ai |
| 85 | gpt-5.6-luna_highOpenAI | closed | effort high | 47.4%43.4 to 51.4 | not dated | conceptualreasoning.ai |
| 86 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 47.1%43.2 to 51.1 | not dated | conceptualreasoning.ai |
| 87 | gemini-3.7-flash_lowGoogle DeepMind | closed | effort low | 47.1%43.1 to 51.1 | not dated | conceptualreasoning.ai |
| 88 | deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_unknown, DeepSeek | open | effort unknown | 47.0%43.0 to 51.0 | not dated | conceptualreasoning.ai |
| 89 | claude-sonnet-4-6_maxAnthropic | closed | effort max | 46.5%42.5 to 50.5 | not dated | conceptualreasoning.ai |
| 90 | gpt-5.6-terra_noneOpenAI | closed | effort none | 46.5%42.5 to 50.5 | not dated | conceptualreasoning.ai |
| 91 | qwen3.8-max_xhighAlibaba | closed | effort xhigh | 46.2%42.2 to 50.3 | not dated | conceptualreasoning.ai |
| 92 | zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI) | open | effort max | 45.8%41.8 to 49.9 | not dated | conceptualreasoning.ai |
| 93 | claude-haiku-5-5_mediumAnthropic | closed | effort medium | 45.6%41.5 to 49.7 | not dated | conceptualreasoning.ai |
| 94 | deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_unknown, DeepSeek | open | effort unknown | 45.5%41.4 to 49.5 | not dated | conceptualreasoning.ai |
| 95 | claude-haiku-5-5_highAnthropic | closed | effort high | 45.2%41.1 to 49.3 | not dated | conceptualreasoning.ai |
| 96 | grok-4.5_highxAI | closed | effort high | 45.2%41.1 to 49.3 | not dated | conceptualreasoning.ai |
| 97 | gemini-3.6-flash_highGoogle DeepMind | closed | effort high | 44.9%40.8 to 49.0 | not dated | conceptualreasoning.ai |
| 98 | claude-haiku-5-5_lowAnthropic | closed | effort low | 44.8%40.7 to 48.9 | not dated | conceptualreasoning.ai |
| 99 | gpt-6-luna_maxOpenAI | closed | effort max | 44.5%40.4 to 48.6 | not dated | conceptualreasoning.ai |
| 100 | claude-opus-4-5-20251101_highAnthropic | closed | effort high | 44.5%40.3 to 48.6 | not dated | conceptualreasoning.ai |
| 101 | gpt-5.6-luna_mediumOpenAI | closed | effort medium | 44.3%40.1 to 48.4 | not dated | conceptualreasoning.ai |
| 102 | qwen3.7-maxAlibaba | closed | default | 44.0%39.9 to 48.2 | not dated | conceptualreasoning.ai |
| 103 | gpt-5.2-2025-12-11_xhighOpenAI | closed | effort xhigh | 43.9%39.8 to 48.0 | not dated | conceptualreasoning.ai |
| 104 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 43.9%39.7 to 48.0 | not dated | conceptualreasoning.ai |
| 105 | gpt-5.6-luna_lowOpenAI | closed | effort low | 43.8%39.6 to 47.9 | not dated | conceptualreasoning.ai |
| 106 | gemini-3-flash-preview_highGoogle DeepMind | closed | effort high | 43.1%39.0 to 47.3 | not dated | conceptualreasoning.ai |
| 107 | qwen3.6-max-previewAlibaba | not stated | default | 42.5%38.3 to 46.7 | not dated | conceptualreasoning.ai |
| 108 | deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeek | open | effort max | 41.7%37.5 to 46.0 | not dated | conceptualreasoning.ai |
| 109 | gpt-5.6-luna_noneOpenAI | closed | effort none | 41.4%37.1 to 45.7 | not dated | conceptualreasoning.ai |
| 110 | Qwen/Qwen3.8-27BEpoch: qwen3.8-27b, Alibaba | open | default | 41.4%37.1 to 45.6 | not dated | conceptualreasoning.ai |
| 111 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeek | open | effort max | 41.2%36.9 to 45.5 | not dated | conceptualreasoning.ai |
| 112 | gpt-5.4-mini-2026-03-17_xhighOpenAI | closed | effort xhigh | 40.8%36.5 to 45.1 | not dated | conceptualreasoning.ai |
| 113 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 40.0%35.7 to 44.3 | not dated | conceptualreasoning.ai |
| 114 | o3-2025-04-16_highOpenAI | closed | effort high | 39.7%35.4 to 44.0 | not dated | conceptualreasoning.ai |
| 115 | claude-haiku-5-5_noneAnthropic | closed | effort none | 39.3%34.9 to 43.6 | not dated | conceptualreasoning.ai |
| 116 | google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMind | open | default | 39.3%34.9 to 43.6 | not dated | conceptualreasoning.ai |
| 117 | claude-sonnet-4-5-20250929_unknownAnthropic | closed | effort unknown | 38.8%34.4 to 43.1 | not dated | conceptualreasoning.ai |
| 118 | grok-4-20xAI | closed | default | 38.7%34.3 to 43.0 | not dated | conceptualreasoning.ai |
| 119 | o3-pro-2025-06-10_highOpenAI | closed | effort high | 38.5%34.1 to 42.9 | not dated | conceptualreasoning.ai |
| 120 | grok-4.3_highxAI | closed | effort high | 38.3%34.0 to 42.7 | not dated | conceptualreasoning.ai |
| 121 | Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibaba | open | default | 37.9%33.5 to 42.3 | not dated | conceptualreasoning.ai |
| 122 | gemini-3.5-flash-lite_highGoogle DeepMind | closed | effort high | 37.6%33.2 to 42.0 | not dated | conceptualreasoning.ai |
| 123 | thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machines | open | effort xhigh | 37.6%33.2 to 42.0 | not dated | conceptualreasoning.ai |
| 124 | qwen3.7-plusAlibaba | closed | default | 37.6%33.2 to 42.0 | not dated | conceptualreasoning.ai |
| 125 | claude-opus-4-20250514_unknownAnthropic | closed | effort unknown | 37.4%33.0 to 41.8 | not dated | conceptualreasoning.ai |
| 126 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 37.3%32.8 to 41.7 | not dated | conceptualreasoning.ai |
| 127 | claude-opus-4-1-20250805_unknownAnthropic | closed | effort unknown | 37.1%32.7 to 41.5 | not dated | conceptualreasoning.ai |
| 128 | nemotron-3-ultraNvidiano page match | open | default | 36.9%32.5 to 41.3 | not dated | conceptualreasoning.ai |
| 129 | gpt-5.4-nano-2026-03-17_xhighOpenAI | closed | effort xhigh | 36.9%32.4 to 41.3 | not dated | conceptualreasoning.ai |
| 130 | qwen3.5-plusAlibaba | not stated | default | 36.4%31.9 to 40.8 | not dated | conceptualreasoning.ai |
| 131 | deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_max, DeepSeek | open | effort max | 35.9%31.4 to 40.4 | not dated | conceptualreasoning.ai |
| 132 | mistral-large-4-0_unknownMistral AIno page match | open | effort unknown | 35.8%31.3 to 40.3 | not dated | conceptualreasoning.ai |
| 133 | gemini-3.1-flash-lite_highGoogle | closed | effort high | 35.0%30.5 to 39.5 | not dated | conceptualreasoning.ai |
| 134 | gemini-2.5-proGoogle DeepMind | closed | default | 34.8%30.3 to 39.3 | not dated | conceptualreasoning.ai |
| 135 | Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibaba | open | default | 34.5%30.0 to 39.0 | not dated | conceptualreasoning.ai |
| 136 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 34.2%29.7 to 38.7 | not dated | conceptualreasoning.ai |
| 137 | Qwen/Qwen3.5-27BEpoch: qwen3.5-27B, Alibaba | open | default | 34.0%29.4 to 38.5 | not dated | conceptualreasoning.ai |
| 138 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMax | open | default | 33.7%29.2 to 38.2 | not dated | conceptualreasoning.ai |
| 139 | qwen3.6-plusAlibaba | closed | default | 33.1%28.5 to 37.7 | not dated | conceptualreasoning.ai |
| 140 | Qwen/Qwen3.5-122B-A10BEpoch: qwen3.5-122b-a10b_none, Alibaba | open | effort none | 32.2%27.6 to 36.8 | not dated | conceptualreasoning.ai |
| 141 | qwen3.6-flashAlibaba | not stated | default | 31.0%26.3 to 35.6 | not dated | conceptualreasoning.ai |
| 142 | claude-haiku-4-5-20251001_unknownAnthropic | closed | effort unknown | 30.9%26.3 to 35.6 | not dated | conceptualreasoning.ai |
| 143 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibaba | open | default | 29.7%25.0 to 34.4 | not dated | conceptualreasoning.ai |
| 144 | gemma-4-26b-a4bGoogle DeepMindno page match | open | default | 29.7%25.0 to 34.4 | not dated | conceptualreasoning.ai |
| 145 | XiaomiMiMo/MiMo-V2.5-ProEpoch: mimo-v2.5-pro, Xiaomi Corp | open | default | 29.5%24.8 to 34.2 | not dated | conceptualreasoning.ai |
| 146 | Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibaba | open | default | 29.5%24.8 to 34.2 | not dated | conceptualreasoning.ai |
| 147 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 29.3%24.6 to 34.0 | not dated | conceptualreasoning.ai |
| 148 | qwen3.5-flashAlibaba | not stated | default | 29.1%24.4 to 33.8 | not dated | conceptualreasoning.ai |
| 149 | deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_thinking, DeepSeek | open | effort thinking | 29.1%24.4 to 33.8 | not dated | conceptualreasoning.ai |
| 150 | claude-sonnet-4-20250514_unknownAnthropic | closed | effort unknown | 29.0%24.3 to 33.7 | not dated | conceptualreasoning.ai |
| 151 | deepseek-ai/DeepSeek-V3.2Epoch: deepseek/deepseek-v3.2, DeepSeek | open | default | 28.8%24.1 to 33.5 | not dated | conceptualreasoning.ai |
| 152 | deepseek-ai/DeepSeek-V3.1-TerminusEpoch: DeepSeek-V3.1-Terminus, DeepSeek | open | default | 28.6%23.9 to 33.3 | not dated | conceptualreasoning.ai |
| 153 | qwen3-max-2025-09-23Alibaba | closed | default | 28.3%23.5 to 33.0 | not dated | conceptualreasoning.ai |
| 154 | gemini-2.5-flashGoogle DeepMind | closed | default | 27.5%22.7 to 32.2 | not dated | conceptualreasoning.ai |
| 155 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 26.5%21.7 to 31.3 | not dated | conceptualreasoning.ai |
| 156 | mistral-medium-2604Mistral AIno page match | open | default | 26.1%21.3 to 30.8 | not dated | conceptualreasoning.ai |
| 157 | gpt-4.1-2025-04-14OpenAI | closed | default | 25.6%20.8 to 30.4 | not dated | conceptualreasoning.ai |
| 158 | Qwen/Qwen3-235B-A22BEpoch: Qwen3-235B-A22B, Alibaba | open | default | 25.0%20.2 to 29.8 | not dated | conceptualreasoning.ai |
| 159 | Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibaba | open | default | 24.5%19.6 to 29.3 | not dated | conceptualreasoning.ai |
| 160 | deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1_thinking, DeepSeek | open | effort thinking | 24.3%19.5 to 29.1 | not dated | conceptualreasoning.ai |
| 161 | qwen-plus-2025-04-28Alibaba | closed | default | 24.0%19.1 to 28.8 | not dated | conceptualreasoning.ai |
| 162 | Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: Qwen3-235B-A22B-Instruct-2507, Alibaba | open | default | 23.5%18.7 to 28.4 | not dated | conceptualreasoning.ai |
| 163 | Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibaba | open | default | 22.4%17.5 to 27.2 | not dated | conceptualreasoning.ai |
| 164 | o1-2024-12-17_highOpenAI | closed | effort high | 22.3%17.4 to 27.1 | not dated | conceptualreasoning.ai |
| 165 | openai/gpt-oss-120bEpoch: gpt-oss-120b_high, OpenAI | open | effort high | 22.1%17.3 to 27.0 | not dated | conceptualreasoning.ai |
| 166 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 21.1%16.2 to 26.0 | not dated | conceptualreasoning.ai |
| 167 | mistral-small-2603Mistral AIno page match | open | default | 20.6%15.7 to 25.5 | not dated | conceptualreasoning.ai |
| 168 | Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibaba | open | default | 19.5%14.6 to 24.5 | not dated | conceptualreasoning.ai |
| 169 | mistral-medium-2508Mistral AI | closed | default | 19.1%14.1 to 24.0 | not dated | conceptualreasoning.ai |
| 170 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 19.0%14.0 to 23.9 | not dated | conceptualreasoning.ai |
| 171 | Qwen/Qwen3-14BEpoch: qwen3-14b, Alibaba | open | default | 18.2%13.2 to 23.2 | not dated | conceptualreasoning.ai |
| 172 | gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMind | closed | default | 18.1%13.1 to 23.0 | not dated | conceptualreasoning.ai |
| 173 | meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AI | open | default | 17.5%12.6 to 22.5 | not dated | conceptualreasoning.ai |
| 174 | Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibaba | open | default | 17.3%12.3 to 22.3 | not dated | conceptualreasoning.ai |
| 175 | gpt-4-0613OpenAI | closed | default | 17.1%12.1 to 22.0 | not dated | conceptualreasoning.ai |
| 176 | claude-3-opus-20240229Anthropic | closed | default | 17.0%12.0 to 22.0 | not dated | conceptualreasoning.ai |
| 177 | mistral-large-2512Mistral AIno page match | open | default | 16.7%11.7 to 21.7 | not dated | conceptualreasoning.ai |
| 178 | gpt-4o-2024-05-13OpenAI | closed | default | 16.6%11.6 to 21.6 | not dated | conceptualreasoning.ai |
| 179 | meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AI | open | default | 15.9%10.9 to 20.9 | not dated | conceptualreasoning.ai |
| 180 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibaba | open | default | 15.8%10.8 to 20.8 | not dated | conceptualreasoning.ai |
| 181 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 15.5%10.5 to 20.5 | not dated | conceptualreasoning.ai |
| 182 | deepseek-chatDeepSeekno page match | open | default | 15.2%10.2 to 20.3 | not dated | conceptualreasoning.ai |
| 183 | meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AI | open | default | 14.8%9.8 to 19.9 | not dated | conceptualreasoning.ai |
| 184 | openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAI | open | effort high | 14.5%9.5 to 19.5 | not dated | conceptualreasoning.ai |
| 185 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 13.4%8.4 to 18.5 | not dated | conceptualreasoning.ai |
| 186 | google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMind | open | default | 12.3%7.2 to 17.3 | not dated | conceptualreasoning.ai |
| 187 | meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AI | open | default | 12.0%6.9 to 17.0 | not dated | conceptualreasoning.ai |
| 188 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 10.4%5.3 to 15.4 | not dated | conceptualreasoning.ai |
| 189 | CohereLabs/c4ai-command-a-03-2025Epoch: c4ai-command-a-03-2025, Cohere | open | default | 10.3%5.2 to 15.4 | not dated | conceptualreasoning.ai |
| 190 | gpt-4-turboOpenAI | closed | default | 9.8%4.7 to 14.9 | not dated | conceptualreasoning.ai |
| 191 | gpt-3.5-turbo-0125OpenAI | closed | default | 9.7%4.6 to 14.7 | not dated | conceptualreasoning.ai |
| 192 | CohereLabs/c4ai-command-r-08-2024Epoch: c4ai-command-r-08-2024 | open | default | 9.2%4.1 to 14.3 | not dated | conceptualreasoning.ai |
| 193 | claude-3-haiku-20240307Anthropic | closed | default | 8.8%3.8 to 13.9 | not dated | conceptualreasoning.ai |
| 194 | Qwen/Qwen3-8BEpoch: qwen3-8b, Alibaba | open | default | 8.8%3.7 to 13.9 | not dated | conceptualreasoning.ai |
| 195 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 7.9%2.8 to 13.0 | not dated | conceptualreasoning.ai |
| 196 | gemma-2-27bGoogle DeepMindno page match | open | default | 7.1%2.0 to 12.2 | not dated | conceptualreasoning.ai |
| 197 | Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibaba | open | default | 6.4%1.3 to 11.5 | not dated | conceptualreasoning.ai |
| 198 | ministral-3b-2410Mistral AI | closed | default | 5.5%0.4 to 10.6 | not dated | conceptualreasoning.ai |
| 199 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 5.5%0.3 to 10.6 | not dated | conceptualreasoning.ai |
| 200 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | default | 5.4%0.3 to 10.5 | not dated | conceptualreasoning.ai |
| 201 | CohereLabs/c4ai-command-r-plus-08-2024Epoch: c4ai-command-r-plus-08-2024, Cohere,Cohere Labs (formerly Cohere for AI) | open | default | 5.0%-0.0 to 10.1 | not dated | conceptualreasoning.ai |
| 202 | google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMind | open | default | 4.5%-0.3 to 9.2 | not dated | conceptualreasoning.ai |
| 203 | google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMind | open | default | 2.8%error bar left out | not dated | conceptualreasoning.ai |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method