Skip to content
Weights Not Walls
Menu

LMCA

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://conceptualreasoning.ai/lmca
Score shown
Epoch's column "Score", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
Results
203 in all: 59 for open-weight models, 140 for closed ones, 4 where Epoch does not say. 51 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K3.1-Pro, 57.6%, number 44 of 203 results.

LMCA results
#ModelWeightsSettingScoreDateSource
1claude-fable-5-2_xhighAnthropicclosedeffort xhigh70.6%68.0 to 73.2not datedconceptualreasoning.ai
2claude-fable-5-2_unknownAnthropicclosedeffort unknown70.1%67.5 to 72.7not datedconceptualreasoning.ai
3claude-fable-5-2_mediumAnthropicclosedeffort medium70.0%67.4 to 72.6not datedconceptualreasoning.ai
4claude-fable-5-2_highAnthropicclosedeffort high69.9%67.3 to 72.5not datedconceptualreasoning.ai
5claude-fable-5-2_lowAnthropicclosedeffort low69.6%67.0 to 72.3not datedconceptualreasoning.ai
6claude-sonnet-5-5_maxAnthropicclosedeffort max68.3%65.5 to 71.0not datedconceptualreasoning.ai
7claude-opus-5-5_maxAnthropicclosedeffort max68.2%65.5 to 71.0not datedconceptualreasoning.ai
8claude-sonnet-5-5_xhighAnthropicclosedeffort xhigh66.3%63.4 to 69.2not datedconceptualreasoning.ai
9claude-opus-5-5_xhighAnthropicclosedeffort xhigh66.2%63.3 to 69.1not datedconceptualreasoning.ai
10claude-sonnet-5-5_highAnthropicclosedeffort high65.9%63.0 to 68.8not datedconceptualreasoning.ai
11claude-opus-5-5_highAnthropicclosedeffort high65.9%63.0 to 68.8not datedconceptualreasoning.ai
12claude-opus-5-5_mediumAnthropicclosedeffort medium65.7%62.7 to 68.6not datedconceptualreasoning.ai
13claude-fable-5-1_unknownAnthropicclosedeffort unknown65.5%62.5 to 68.4not datedconceptualreasoning.ai
14claude-sonnet-5-5_mediumAnthropicclosedeffort medium65.5%62.5 to 68.4not datedconceptualreasoning.ai
15claude-fable-5-1_mediumAnthropicclosedeffort medium65.3%62.4 to 68.3not datedconceptualreasoning.ai
16claude-fable-5-1_highAnthropicclosedeffort high65.1%62.1 to 68.0not datedconceptualreasoning.ai
17claude-fable-5-1_xhighAnthropicclosedeffort xhigh65.1%62.1 to 68.0not datedconceptualreasoning.ai
18claude-opus-5-5_lowAnthropicclosedeffort low64.9%62.0 to 67.9not datedconceptualreasoning.ai
19claude-sonnet-5-5_lowAnthropicclosedeffort low64.5%61.5 to 67.5not datedconceptualreasoning.ai
20claude-opus-5_xhighAnthropicclosedeffort xhigh64.5%61.5 to 67.5not datedconceptualreasoning.ai
21gpt-6-astra_xhighOpenAIclosedeffort xhigh64.4%61.4 to 67.4not datedconceptualreasoning.ai
22gpt-6-astra_unknownOpenAIclosedeffort unknown64.1%61.1 to 67.1not datedconceptualreasoning.ai
23gpt-6-astra_highOpenAIclosedeffort high63.8%60.8 to 66.9not datedconceptualreasoning.ai
24gemini-4-argon_highGoogle DeepMindclosedeffort high63.7%60.6 to 66.7not datedconceptualreasoning.ai
25claude-opus-5_highAnthropicclosedeffort high63.7%60.6 to 66.7not datedconceptualreasoning.ai
26claude-fable-5-1_lowAnthropicclosedeffort low63.5%60.4 to 66.5not datedconceptualreasoning.ai
27claude-opus-5_maxAnthropicclosedeffort max63.3%60.2 to 66.4not datedconceptualreasoning.ai
28claude-opus-5_mediumAnthropicclosedeffort medium62.9%59.8 to 66.0not datedconceptualreasoning.ai
29gpt-6-astra_mediumOpenAIclosedeffort medium62.9%59.8 to 66.0not datedconceptualreasoning.ai
30gemini-4-argon_lowGoogle DeepMindclosedeffort low62.9%59.8 to 66.0not datedconceptualreasoning.ai
31gemini-4-argon_mediumGoogle DeepMindclosedeffort medium62.8%59.7 to 66.0not datedconceptualreasoning.ai
32gpt-6.1-sol_maxOpenAIclosedeffort max62.8%59.7 to 65.9not datedconceptualreasoning.ai
33claude-opus-5_lowAnthropicclosedeffort low62.6%59.5 to 65.7not datedconceptualreasoning.ai
34gpt-6-astra_lowOpenAIclosedeffort low62.2%59.0 to 65.3not datedconceptualreasoning.ai
35claude-fable-5_highAnthropicclosedeffort high61.2%57.9 to 64.4not datedconceptualreasoning.ai
36claude-fable-5_xhighAnthropicclosedeffort xhigh61.0%57.8 to 64.2not datedconceptualreasoning.ai
37claude-fable-5_maxAnthropicclosedeffort max60.3%57.1 to 63.6not datedconceptualreasoning.ai
38claude-fable-5_mediumAnthropicclosedeffort medium60.3%57.1 to 63.6not datedconceptualreasoning.ai
39claude-fable-5_lowAnthropicclosedeffort low59.6%56.3 to 62.9not datedconceptualreasoning.ai
40gpt-5.6-sol_promaxOpenAIcloseddefault59.2%55.9 to 62.5not datedconceptualreasoning.ai
41gpt-6-sol_maxOpenAIclosedeffort max59.1%55.7 to 62.4not datedconceptualreasoning.ai
42gpt-5.6-sol_xhighOpenAIclosedeffort xhigh58.5%55.1 to 61.9not datedconceptualreasoning.ai
43gpt-5.6-sol_maxOpenAIclosedeffort max58.4%55.0 to 61.8not datedconceptualreasoning.ai
44moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro_max, Moonshotopeneffort max57.6%54.2 to 61.0not datedconceptualreasoning.ai
45claude-opus-4-8_maxAnthropicclosedeffort max57.5%54.1 to 60.9not datedconceptualreasoning.ai
46gpt-5.6-sol_highOpenAIclosedeffort high56.9%53.5 to 60.4not datedconceptualreasoning.ai
47muse-wonder-1.0_unknownMeta AIclosedeffort unknown56.8%53.4 to 60.3not datedconceptualreasoning.ai
48gpt-5.6-sol_mediumOpenAIclosedeffort medium56.4%52.9 to 59.9not datedconceptualreasoning.ai
49gpt-5.6-sol_lowOpenAIclosedeffort low55.9%52.4 to 59.4not datedconceptualreasoning.ai
50claude-opus-4-6_maxAnthropicclosedeffort max55.8%52.2 to 59.3not datedconceptualreasoning.ai
51zai-org/GLM-5.3Epoch: glm-5.3_unknown, Z.ai (Zhipu AI)openeffort unknown55.5%52.0 to 59.1not datedconceptualreasoning.ai
52gpt-5.6-terra_maxOpenAIclosedeffort max54.9%51.4 to 58.5not datedconceptualreasoning.ai
53gpt-5.5_xhighOpenAIclosedeffort xhigh54.3%50.6 to 57.9not datedconceptualreasoning.ai
54gpt-5.5-pro_xhighOpenAIclosedeffort xhigh53.9%50.3 to 57.6not datedconceptualreasoning.ai
55muse-spark-1.3_unknownMeta AIclosedeffort unknown53.9%50.2 to 57.5not datedconceptualreasoning.ai
56gemini-3.1-pro-preview_highGoogle DeepMindclosedeffort high53.8%50.2 to 57.5not datedconceptualreasoning.ai
57gemini-3.8-flash_mediumGoogle DeepMindclosedeffort medium52.9%49.2 to 56.6not datedconceptualreasoning.ai
58gemini-3.1-pro-preview_mediumGoogle DeepMindclosedeffort medium52.8%49.1 to 56.5not datedconceptualreasoning.ai
59gpt-5.6-sol_noneOpenAIclosedeffort none52.8%49.0 to 56.5not datedconceptualreasoning.ai
60moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshotopeneffort max52.7%49.0 to 56.4not datedconceptualreasoning.ai
61gpt-5.6-terra_xhighOpenAIclosedeffort xhigh52.2%48.5 to 56.0not datedconceptualreasoning.ai
62claude-opus-4-7_maxAnthropicclosedeffort max52.2%48.5 to 55.9not datedconceptualreasoning.ai
63gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh52.0%48.2 to 55.7not datedconceptualreasoning.ai
64gemini-3.8-flash_unknownGoogle DeepMindclosedeffort unknown51.5%47.8 to 55.3not datedconceptualreasoning.ai
65gpt-5.6-terra_highOpenAIclosedeffort high51.4%47.6 to 55.1not datedconceptualreasoning.ai
66gemini-3.1-pro-preview_lowGoogle DeepMindclosedeffort low51.2%47.4 to 55.0not datedconceptualreasoning.ai
67gemini-3.8-flash_lowGoogle DeepMindclosedeffort low51.2%47.4 to 54.9not datedconceptualreasoning.ai
68claude-haiku-5-5_maxAnthropicclosedeffort max51.1%47.3 to 54.9not datedconceptualreasoning.ai
69gemini-3.7-flash_highGoogle DeepMindclosedeffort high50.4%46.6 to 54.2not datedconceptualreasoning.ai
70claude-sonnet-5_highAnthropicclosedeffort high50.0%46.2 to 53.9not datedconceptualreasoning.ai
71muse-spark-1.1_highMeta AIclosedeffort high49.9%46.1 to 53.8not datedconceptualreasoning.ai
72claude-sonnet-5_mediumAnthropicclosedeffort medium49.7%45.8 to 53.6not datedconceptualreasoning.ai
73claude-sonnet-5_lowAnthropicclosedeffort low49.6%45.8 to 53.5not datedconceptualreasoning.ai
74gpt-5.6-terra_mediumOpenAIclosedeffort medium49.6%45.8 to 53.5not datedconceptualreasoning.ai
75grok-4.7_xhighxAIclosedeffort xhigh49.4%45.5 to 53.3not datedconceptualreasoning.ai
76claude-sonnet-5_maxAnthropicclosedeffort max49.3%45.4 to 53.2not datedconceptualreasoning.ai
77gpt-5.6-terra_lowOpenAIclosedeffort low49.3%45.4 to 53.2not datedconceptualreasoning.ai
78gemini-3.7-flash_mediumGoogle DeepMindclosedeffort medium49.2%45.3 to 53.1not datedconceptualreasoning.ai
79claude-sonnet-5_xhighAnthropicclosedeffort xhigh48.8%44.9 to 52.7not datedconceptualreasoning.ai
80gpt-5.6-luna_maxOpenAIclosedeffort max48.5%44.6 to 52.5not datedconceptualreasoning.ai
81gpt-5.6-luna_xhighOpenAIclosedeffort xhigh48.5%44.6 to 52.5not datedconceptualreasoning.ai
82grok-4.6_xhighxAIclosedeffort xhigh48.4%44.5 to 52.4not datedconceptualreasoning.ai
83muse-spark-1.2_xhighMeta AIclosedeffort xhigh48.4%44.4 to 52.3not datedconceptualreasoning.ai
84claude-haiku-5-5_xhighAnthropicclosedeffort xhigh48.0%44.0 to 51.9not datedconceptualreasoning.ai
85gpt-5.6-luna_highOpenAIclosedeffort high47.4%43.4 to 51.4not datedconceptualreasoning.ai
86gemini-3.5-flash_highGoogle DeepMindclosedeffort high47.1%43.2 to 51.1not datedconceptualreasoning.ai
87gemini-3.7-flash_lowGoogle DeepMindclosedeffort low47.1%43.1 to 51.1not datedconceptualreasoning.ai
88deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_unknown, DeepSeekopeneffort unknown47.0%43.0 to 51.0not datedconceptualreasoning.ai
89claude-sonnet-4-6_maxAnthropicclosedeffort max46.5%42.5 to 50.5not datedconceptualreasoning.ai
90gpt-5.6-terra_noneOpenAIclosedeffort none46.5%42.5 to 50.5not datedconceptualreasoning.ai
91qwen3.8-max_xhighAlibabaclosedeffort xhigh46.2%42.2 to 50.3not datedconceptualreasoning.ai
92zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI)openeffort max45.8%41.8 to 49.9not datedconceptualreasoning.ai
93claude-haiku-5-5_mediumAnthropicclosedeffort medium45.6%41.5 to 49.7not datedconceptualreasoning.ai
94deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_unknown, DeepSeekopeneffort unknown45.5%41.4 to 49.5not datedconceptualreasoning.ai
95claude-haiku-5-5_highAnthropicclosedeffort high45.2%41.1 to 49.3not datedconceptualreasoning.ai
96grok-4.5_highxAIclosedeffort high45.2%41.1 to 49.3not datedconceptualreasoning.ai
97gemini-3.6-flash_highGoogle DeepMindclosedeffort high44.9%40.8 to 49.0not datedconceptualreasoning.ai
98claude-haiku-5-5_lowAnthropicclosedeffort low44.8%40.7 to 48.9not datedconceptualreasoning.ai
99gpt-6-luna_maxOpenAIclosedeffort max44.5%40.4 to 48.6not datedconceptualreasoning.ai
100claude-opus-4-5-20251101_highAnthropicclosedeffort high44.5%40.3 to 48.6not datedconceptualreasoning.ai
101gpt-5.6-luna_mediumOpenAIclosedeffort medium44.3%40.1 to 48.4not datedconceptualreasoning.ai
102qwen3.7-maxAlibabacloseddefault44.0%39.9 to 48.2not datedconceptualreasoning.ai
103gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh43.9%39.8 to 48.0not datedconceptualreasoning.ai
104gpt-5.1-2025-11-13_highOpenAIclosedeffort high43.9%39.7 to 48.0not datedconceptualreasoning.ai
105gpt-5.6-luna_lowOpenAIclosedeffort low43.8%39.6 to 47.9not datedconceptualreasoning.ai
106gemini-3-flash-preview_highGoogle DeepMindclosedeffort high43.1%39.0 to 47.3not datedconceptualreasoning.ai
107qwen3.6-max-previewAlibabanot stateddefault42.5%38.3 to 46.7not datedconceptualreasoning.ai
108deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeekopeneffort max41.7%37.5 to 46.0not datedconceptualreasoning.ai
109gpt-5.6-luna_noneOpenAIclosedeffort none41.4%37.1 to 45.7not datedconceptualreasoning.ai
110Qwen/Qwen3.8-27BEpoch: qwen3.8-27b, Alibabaopendefault41.4%37.1 to 45.6not datedconceptualreasoning.ai
111deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeekopeneffort max41.2%36.9 to 45.5not datedconceptualreasoning.ai
112gpt-5.4-mini-2026-03-17_xhighOpenAIclosedeffort xhigh40.8%36.5 to 45.1not datedconceptualreasoning.ai
113gpt-5-2025-08-07_highOpenAIclosedeffort high40.0%35.7 to 44.3not datedconceptualreasoning.ai
114o3-2025-04-16_highOpenAIclosedeffort high39.7%35.4 to 44.0not datedconceptualreasoning.ai
115claude-haiku-5-5_noneAnthropicclosedeffort none39.3%34.9 to 43.6not datedconceptualreasoning.ai
116google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMindopendefault39.3%34.9 to 43.6not datedconceptualreasoning.ai
117claude-sonnet-4-5-20250929_unknownAnthropicclosedeffort unknown38.8%34.4 to 43.1not datedconceptualreasoning.ai
118grok-4-20xAIcloseddefault38.7%34.3 to 43.0not datedconceptualreasoning.ai
119o3-pro-2025-06-10_highOpenAIclosedeffort high38.5%34.1 to 42.9not datedconceptualreasoning.ai
120grok-4.3_highxAIclosedeffort high38.3%34.0 to 42.7not datedconceptualreasoning.ai
121Qwen/Qwen3.5-397B-A17BEpoch: qwen3.5-397b-a17b, Alibabaopendefault37.9%33.5 to 42.3not datedconceptualreasoning.ai
122gemini-3.5-flash-lite_highGoogle DeepMindclosedeffort high37.6%33.2 to 42.0not datedconceptualreasoning.ai
123thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machinesopeneffort xhigh37.6%33.2 to 42.0not datedconceptualreasoning.ai
124qwen3.7-plusAlibabacloseddefault37.6%33.2 to 42.0not datedconceptualreasoning.ai
125claude-opus-4-20250514_unknownAnthropicclosedeffort unknown37.4%33.0 to 41.8not datedconceptualreasoning.ai
126moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault37.3%32.8 to 41.7not datedconceptualreasoning.ai
127claude-opus-4-1-20250805_unknownAnthropicclosedeffort unknown37.1%32.7 to 41.5not datedconceptualreasoning.ai
128nemotron-3-ultraNvidiano page matchopendefault36.9%32.5 to 41.3not datedconceptualreasoning.ai
129gpt-5.4-nano-2026-03-17_xhighOpenAIclosedeffort xhigh36.9%32.4 to 41.3not datedconceptualreasoning.ai
130qwen3.5-plusAlibabanot stateddefault36.4%31.9 to 40.8not datedconceptualreasoning.ai
131deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_max, DeepSeekopeneffort max35.9%31.4 to 40.4not datedconceptualreasoning.ai
132mistral-large-4-0_unknownMistral AIno page matchopeneffort unknown35.8%31.3 to 40.3not datedconceptualreasoning.ai
133gemini-3.1-flash-lite_highGoogleclosedeffort high35.0%30.5 to 39.5not datedconceptualreasoning.ai
134gemini-2.5-proGoogle DeepMindcloseddefault34.8%30.3 to 39.3not datedconceptualreasoning.ai
135Qwen/Qwen3.6-27BEpoch: qwen3.6-27b, Alibabaopendefault34.5%30.0 to 39.0not datedconceptualreasoning.ai
136gpt-5-mini-2025-08-07_highOpenAIclosedeffort high34.2%29.7 to 38.7not datedconceptualreasoning.ai
137Qwen/Qwen3.5-27BEpoch: qwen3.5-27B, Alibabaopendefault34.0%29.4 to 38.5not datedconceptualreasoning.ai
138MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMaxopendefault33.7%29.2 to 38.2not datedconceptualreasoning.ai
139qwen3.6-plusAlibabacloseddefault33.1%28.5 to 37.7not datedconceptualreasoning.ai
140Qwen/Qwen3.5-122B-A10BEpoch: qwen3.5-122b-a10b_none, Alibabaopeneffort none32.2%27.6 to 36.8not datedconceptualreasoning.ai
141qwen3.6-flashAlibabanot stateddefault31.0%26.3 to 35.6not datedconceptualreasoning.ai
142claude-haiku-4-5-20251001_unknownAnthropicclosedeffort unknown30.9%26.3 to 35.6not datedconceptualreasoning.ai
143Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibabaopendefault29.7%25.0 to 34.4not datedconceptualreasoning.ai
144gemma-4-26b-a4bGoogle DeepMindno page matchopendefault29.7%25.0 to 34.4not datedconceptualreasoning.ai
145XiaomiMiMo/MiMo-V2.5-ProEpoch: mimo-v2.5-pro, Xiaomi Corpopendefault29.5%24.8 to 34.2not datedconceptualreasoning.ai
146Qwen/Qwen3.5-35B-A3BEpoch: qwen3.5-35b-a3b, Alibabaopendefault29.5%24.8 to 34.2not datedconceptualreasoning.ai
147Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault29.3%24.6 to 34.0not datedconceptualreasoning.ai
148qwen3.5-flashAlibabanot stateddefault29.1%24.4 to 33.8not datedconceptualreasoning.ai
149deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_thinking, DeepSeekopeneffort thinking29.1%24.4 to 33.8not datedconceptualreasoning.ai
150claude-sonnet-4-20250514_unknownAnthropicclosedeffort unknown29.0%24.3 to 33.7not datedconceptualreasoning.ai
151deepseek-ai/DeepSeek-V3.2Epoch: deepseek/deepseek-v3.2, DeepSeekopendefault28.8%24.1 to 33.5not datedconceptualreasoning.ai
152deepseek-ai/DeepSeek-V3.1-TerminusEpoch: DeepSeek-V3.1-Terminus, DeepSeekopendefault28.6%23.9 to 33.3not datedconceptualreasoning.ai
153qwen3-max-2025-09-23Alibabacloseddefault28.3%23.5 to 33.0not datedconceptualreasoning.ai
154gemini-2.5-flashGoogle DeepMindcloseddefault27.5%22.7 to 32.2not datedconceptualreasoning.ai
155o4-mini-2025-04-16_highOpenAIclosedeffort high26.5%21.7 to 31.3not datedconceptualreasoning.ai
156mistral-medium-2604Mistral AIno page matchopendefault26.1%21.3 to 30.8not datedconceptualreasoning.ai
157gpt-4.1-2025-04-14OpenAIcloseddefault25.6%20.8 to 30.4not datedconceptualreasoning.ai
158Qwen/Qwen3-235B-A22BEpoch: Qwen3-235B-A22B, Alibabaopendefault25.0%20.2 to 29.8not datedconceptualreasoning.ai
159Qwen/Qwen3.5-9BEpoch: qwen3.5-9B, Alibabaopendefault24.5%19.6 to 29.3not datedconceptualreasoning.ai
160deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1_thinking, DeepSeekopeneffort thinking24.3%19.5 to 29.1not datedconceptualreasoning.ai
161qwen-plus-2025-04-28Alibabacloseddefault24.0%19.1 to 28.8not datedconceptualreasoning.ai
162Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: Qwen3-235B-A22B-Instruct-2507, Alibabaopendefault23.5%18.7 to 28.4not datedconceptualreasoning.ai
163Qwen/Qwen3-30B-A3B-Instruct-2507Epoch: qwen3-30b-a3b-instruct-2507, Alibabaopendefault22.4%17.5 to 27.2not datedconceptualreasoning.ai
164o1-2024-12-17_highOpenAIclosedeffort high22.3%17.4 to 27.1not datedconceptualreasoning.ai
165openai/gpt-oss-120bEpoch: gpt-oss-120b_high, OpenAIopeneffort high22.1%17.3 to 27.0not datedconceptualreasoning.ai
166gpt-4.1-mini-2025-04-14OpenAIcloseddefault21.1%16.2 to 26.0not datedconceptualreasoning.ai
167mistral-small-2603Mistral AIno page matchopendefault20.6%15.7 to 25.5not datedconceptualreasoning.ai
168Qwen/Qwen3-30B-A3B-Thinking-2507Epoch: qwen3-30b-a3b-thinking-2507, Alibabaopendefault19.5%14.6 to 24.5not datedconceptualreasoning.ai
169mistral-medium-2508Mistral AIcloseddefault19.1%14.1 to 24.0not datedconceptualreasoning.ai
170o3-mini-2025-01-31_highOpenAIclosedeffort high19.0%14.0 to 23.9not datedconceptualreasoning.ai
171Qwen/Qwen3-14BEpoch: qwen3-14b, Alibabaopendefault18.2%13.2 to 23.2not datedconceptualreasoning.ai
172gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMindcloseddefault18.1%13.1 to 23.0not datedconceptualreasoning.ai
173meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopendefault17.5%12.6 to 22.5not datedconceptualreasoning.ai
174Qwen/Qwen3-32BEpoch: Qwen3-32B, Alibabaopendefault17.3%12.3 to 22.3not datedconceptualreasoning.ai
175gpt-4-0613OpenAIcloseddefault17.1%12.1 to 22.0not datedconceptualreasoning.ai
176claude-3-opus-20240229Anthropiccloseddefault17.0%12.0 to 22.0not datedconceptualreasoning.ai
177mistral-large-2512Mistral AIno page matchopendefault16.7%11.7 to 21.7not datedconceptualreasoning.ai
178gpt-4o-2024-05-13OpenAIcloseddefault16.6%11.6 to 21.6not datedconceptualreasoning.ai
179meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AIopendefault15.9%10.9 to 20.9not datedconceptualreasoning.ai
180Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibabaopendefault15.8%10.8 to 20.8not datedconceptualreasoning.ai
181deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeekopendefault15.5%10.5 to 20.5not datedconceptualreasoning.ai
182deepseek-chatDeepSeekno page matchopendefault15.2%10.2 to 20.3not datedconceptualreasoning.ai
183meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AIopendefault14.8%9.8 to 19.9not datedconceptualreasoning.ai
184openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAIopeneffort high14.5%9.5 to 19.5not datedconceptualreasoning.ai
185Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibabaopendefault13.4%8.4 to 18.5not datedconceptualreasoning.ai
186google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMindopendefault12.3%7.2 to 17.3not datedconceptualreasoning.ai
187meta-llama/Llama-4-Scout-17B-16E-InstructEpoch: Llama-4-Scout-17B-16E-Instruct, Meta AIopendefault12.0%6.9 to 17.0not datedconceptualreasoning.ai
188gpt-4o-mini-2024-07-18OpenAIcloseddefault10.4%5.3 to 15.4not datedconceptualreasoning.ai
189CohereLabs/c4ai-command-a-03-2025Epoch: c4ai-command-a-03-2025, Cohereopendefault10.3%5.2 to 15.4not datedconceptualreasoning.ai
190gpt-4-turboOpenAIcloseddefault9.8%4.7 to 14.9not datedconceptualreasoning.ai
191gpt-3.5-turbo-0125OpenAIcloseddefault9.7%4.6 to 14.7not datedconceptualreasoning.ai
192CohereLabs/c4ai-command-r-08-2024Epoch: c4ai-command-r-08-2024opendefault9.2%4.1 to 14.3not datedconceptualreasoning.ai
193claude-3-haiku-20240307Anthropiccloseddefault8.8%3.8 to 13.9not datedconceptualreasoning.ai
194Qwen/Qwen3-8BEpoch: qwen3-8b, Alibabaopendefault8.8%3.7 to 13.9not datedconceptualreasoning.ai
195gpt-5-nano-2025-08-07_highOpenAIclosedeffort high7.9%2.8 to 13.0not datedconceptualreasoning.ai
196gemma-2-27bGoogle DeepMindno page matchopendefault7.1%2.0 to 12.2not datedconceptualreasoning.ai
197Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, Alibabaopendefault6.4%1.3 to 11.5not datedconceptualreasoning.ai
198ministral-3b-2410Mistral AIcloseddefault5.5%0.4 to 10.6not datedconceptualreasoning.ai
199gpt-4.1-nano-2025-04-14OpenAIcloseddefault5.5%0.3 to 10.6not datedconceptualreasoning.ai
200meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopendefault5.4%0.3 to 10.5not datedconceptualreasoning.ai
201CohereLabs/c4ai-command-r-plus-08-2024Epoch: c4ai-command-r-plus-08-2024, Cohere,Cohere Labs (formerly Cohere for AI)opendefault5.0%-0.0 to 10.1not datedconceptualreasoning.ai
202google/gemma-3-12b-itEpoch: gemma-3-12b-it, Google DeepMindopendefault4.5%-0.3 to 9.2not datedconceptualreasoning.ai
203google/gemma-3-4b-itEpoch: gemma-3-4b-it, Google DeepMindopendefault2.8%error bar left outnot datedconceptualreasoning.ai

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method