Skip to content
Weights Not Walls
Menu

WeirdML

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://htihle.github.io/weirdml.html
Score shown
Epoch's column "Accuracy", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
Results
173 in all: 58 for open-weight models, 114 for closed ones, 1 where Epoch does not say. 53 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K3, 82.6%, number 18 of 173 results.

WeirdML results
#ModelWeightsSettingScoreDateSource
1gpt-6-astra_promaxOpenAIcloseddefault93.6%±0.0not datedhtihle.github.io
2gpt-6-astra_maxOpenAIclosedeffort max93.3%±0.0not datedhtihle.github.io
3claude-fable-5-1_maxAnthropicclosedeffort max92.9%±0.0not datedhtihle.github.io
4gpt-6-astra_highOpenAIclosedeffort high92.9%±0.0not datedhtihle.github.io
5claude-fable-5-1_highAnthropicclosedeffort high92.3%±0.0not datedhtihle.github.io
6claude-fable-5_maxAnthropicclosedeffort max91.9%±0.0not datedhtihle.github.io
7claude-opus-5_maxAnthropicclosedeffort max91.8%±0.0not datedhtihle.github.io
8claude-opus-5_highAnthropicclosedeffort high91.6%±0.0not datedhtihle.github.io
9gpt-5.6-sol_promaxOpenAIcloseddefault89.4%±0.0not datedhtihle.github.io
10gpt-5.6-sol_highOpenAIclosedeffort high88.8%±0.0not datedhtihle.github.io
11claude-fable-5_highAnthropicclosedeffort high87.8%±0.0not datedWeirdML Leaderboard, via Epoch AI
12gpt-5.6-sol_maxOpenAIclosedeffort max87.0%±0.0not datedhtihle.github.io
13claude-opus-5Anthropiccloseddefault86.3%±0.0not datedhtihle.github.io
14gpt-5.5_xhighOpenAIclosedeffort xhigh84.9%±0.0not datedhtihle.github.io
15gemini-3.8-flash_highGoogle DeepMindclosedeffort high84.8%±0.0not datedhtihle.github.io
16gpt-5.5_highOpenAIclosedeffort high83.9%±0.0not datedhtihle.github.io
17claude-opus-4-8_xhighAnthropicclosedeffort xhigh82.9%±0.0not datedhtihle.github.io
18moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshotopeneffort max82.6%±0.0not datedhtihle.github.io
19gpt-5.3-codexOpenAIcloseddefault79.3%not datedWeirdML Leaderboard, via Epoch AI
20gpt-5.6-terra_highOpenAIclosedeffort high78.3%±0.0not datedhtihle.github.io
21claude-opus-4-6_highAnthropicclosedeffort high78.0%±0.0not datedhtihle.github.io
22claude-opus-4-6_unknownAnthropicclosedeffort unknown77.9%not datedWeirdML Leaderboard, via Epoch AI
23gpt-5.3-codex_xhighOpenAIclosedeffort xhigh77.9%±0.0not datedhtihle.github.io
24gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh77.7%±0.0not datedhtihle.github.io
25claude-opus-4-7_highAnthropicclosedeffort high76.4%±0.0not datedhtihle.github.io
26claude-opus-4-7Anthropiccloseddefault76.4%±0.0not datedhtihle.github.io
27claude-opus-4-7_unknownAnthropicclosedeffort unknown76.4%not datedWeirdML Leaderboard, via Epoch AI
28claude-opus-4-8_mediumAnthropicclosedeffort medium76.0%±0.0not datedhtihle.github.io
29claude-opus-4-7_maxAnthropicclosedeffort max75.5%±0.0not datedhtihle.github.io
30zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI)openeffort max75.4%±0.0not datedhtihle.github.io
31gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh72.2%±0.0not datedhtihle.github.io
32gemini-3.1-pro-previewGoogle DeepMindcloseddefault72.1%±0.0not datedhtihle.github.io
33claude-opus-4-8_noneAnthropicclosedeffort none70.5%±0.0not datedhtihle.github.io
34zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI)openeffort max70.1%±0.0not datedhtihle.github.io
35gemini-3-pro-previewGoogle DeepMindcloseddefault69.9%±0.0not datedhtihle.github.io
36claude-sonnet-5_highAnthropicclosedeffort high69.0%±0.0not datedhtihle.github.io
37zai-org/GLM-5.2Epoch: glm-5.2_high, Z.ai (Zhipu AI)openeffort high67.3%±0.0not datedhtihle.github.io
38grok-4.6_highxAIclosedeffort high67.3%±0.0not datedhtihle.github.io
39gpt-5.5_noneOpenAIclosedeffort none67.2%±0.0not datedhtihle.github.io
40deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_max, DeepSeekopeneffort max66.2%±0.0not datedhtihle.github.io
41claude-sonnet-4-6_mediumAnthropicclosedeffort medium66.1%±0.0not datedWeirdML Leaderboard, via Epoch AI
42claude-opus-4-6Anthropiccloseddefault65.9%±0.0not datedhtihle.github.io
43claude-opus-4-5-20251101_16KAnthropiccloseddefault63.7%±0.0not datedhtihle.github.io
44gpt-5.2-2025-12-11_mediumOpenAIclosedeffort medium63.4%±0.0not datedhtihle.github.io
45deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeekopeneffort max63.0%±0.0not datedhtihle.github.io
46gemini-3.5-flash_highGoogle DeepMindclosedeffort high62.6%±0.0not datedhtihle.github.io
47gemini-3-flash-previewGoogle DeepMindcloseddefault61.6%±0.0not datedhtihle.github.io
48gpt-5.6-luna_highOpenAIclosedeffort high60.9%±0.0not datedhtihle.github.io
49gpt-5.1-2025-11-13_highOpenAIclosedeffort high60.8%±0.0not datedhtihle.github.io
50gpt-5-2025-08-07_highOpenAIclosedeffort high60.7%±0.0not datedhtihle.github.io
51gpt-5-pro-2025-10-06_highOpenAIclosedeffort high60.4%±0.0not datedhtihle.github.io
52muse-spark-1.2_xhighMeta AIclosedeffort xhigh60.4%±0.0not datedhtihle.github.io
53gpt-5.4-mini-2026-03-17_highOpenAIclosedeffort high60.3%±0.0not datedhtihle.github.io
54o3-pro-2025-06-10_highOpenAIclosedeffort high58.2%±0.0not datedhtihle.github.io
55gpt-5.4-2026-03-05_noneOpenAIclosedeffort none57.4%±0.0not datedWeirdML Leaderboard, via Epoch AI
56gpt-5.4-pro-2026-03-05_noneOpenAIclosedeffort none57.4%±0.0not datedhtihle.github.io
57zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI)opendefault57.1%±0.0not datedhtihle.github.io
58deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_high, DeepSeekopeneffort high57.0%±0.0not datedhtihle.github.io
59gemini-3.6-flash_highGoogle DeepMindclosedeffort high56.1%±0.0not datedhtihle.github.io
60moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault55.9%±0.0not datedhtihle.github.io
61gpt-5-codexOpenAIcloseddefault54.5%not datedWeirdML Leaderboard
62gpt-5-codex_highOpenAIclosedeffort high54.5%±0.0not datedWeirdML Leaderboard, via Epoch AI
63moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, Moonshotopendefault54.1%±0.0not datedhtihle.github.io
64gemini-2.5-pro_16KGoogle DeepMindcloseddefault54.0%not datedWeirdML Leaderboard
65gpt-5-mini-2025-08-07_highOpenAIclosedeffort high52.7%±0.0not datedhtihle.github.io
66o4-mini-2025-04-16_highOpenAIclosedeffort high52.6%±0.0not datedhtihle.github.io
67o3-2025-04-16_highOpenAIclosedeffort high52.4%±0.0not datedhtihle.github.io
68google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMindopendefault52.3%±0.0not datedhtihle.github.io
69grok-4-20xAIcloseddefault52.3%±0.0not datedhtihle.github.io
70gemini-3.1-flash-liteGooglecloseddefault52.2%±0.0not datedWeirdML Leaderboard, via Epoch AI
71grok-4.3_unknownxAIclosedeffort unknown49.9%±0.0not datedhtihle.github.io
72gpt-5.2-2025-12-11_lowOpenAIclosedeffort low49.6%not datedWeirdML Leaderboard
73gpt-5.2-2025-12-11_noneOpenAIclosedeffort none49.6%±0.0not datedWeirdML Leaderboard, via Epoch AI
74gpt-5.4-nano-2026-03-17_highOpenAIclosedeffort high49.2%±0.0not datedhtihle.github.io
75deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeekopeneffort max48.6%±0.0not datedhtihle.github.io
76zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI)opendefault48.2%±0.0not datedWeirdML Leaderboard, via Epoch AI
77openai/gpt-oss-120bEpoch: gpt-oss-120b_high, OpenAIopeneffort high48.2%not datedWeirdML Leaderboard
78openai/gpt-oss-120bEpoch: openai/gpt-oss-120b_high, OpenAIopeneffort high48.2%±0.0not datedWeirdML Leaderboard, via Epoch AI
79claude-sonnet-4-5-20250929_16KAnthropiccloseddefault47.7%±0.0not datedhtihle.github.io
80o1-preview-2024-09-12OpenAIcloseddefault47.6%±0.0not datedhtihle.github.io
81deepseek-ai/DeepSeek-V3.2-SpecialeEpoch: DeepSeek-V3.2-Speciale, DeepSeekopendefault46.7%±0.0not datedWeirdML Leaderboard, via Epoch AI
82claude-sonnet-4-5-20250929Anthropiccloseddefault46.7%±0.0not datedhtihle.github.io
83deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_high, DeepSeekopeneffort high46.5%±0.0not datedhtihle.github.io
84grok-4.5_unknownxAIclosedeffort unknown46.4%±0.0not datedhtihle.github.io
85claude-sonnet-4-20250514_16KAnthropiccloseddefault46.1%±0.0not datedhtihle.github.io
86o1-2024-12-17_highOpenAIclosedeffort high46.1%±0.0not datedhtihle.github.io
87claude-opus-4-1-20250805_16KAnthropiccloseddefault45.9%±0.0not datedhtihle.github.io
88grok-4-0709xAIcloseddefault45.7%±0.0not datedhtihle.github.io
89deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_max, DeepSeekopeneffort max45.6%±0.0not datedhtihle.github.io
90moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshotopendefault45.6%±0.0not datedhtihle.github.io
91claude-haiku-4-5-20251001Anthropiccloseddefault45.4%±0.0not datedhtihle.github.io
92claude-haiku-4-5-20251001_16KAnthropiccloseddefault44.1%±0.0not datedhtihle.github.io
93claude-sonnet-4-20250514Anthropiccloseddefault43.9%±0.0not datedhtihle.github.io
94deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_high, DeepSeekopeneffort high43.8%±0.0not datedhtihle.github.io
95o3-mini-2025-01-31_highOpenAIclosedeffort high43.8%±0.0not datedhtihle.github.io
96claude-opus-4-20250514_16KAnthropiccloseddefault43.7%±0.0not datedhtihle.github.io
97mistral-medium-2604Mistral AIno page matchopendefault43.7%±0.0not datedhtihle.github.io
98nemotron-3-ultraNvidiano page matchopendefault43.5%±0.0not datedhtihle.github.io
99mercury-2Inception Labscloseddefault43.2%±0.0not datedhtihle.github.io
100grok-4-fastxAIcloseddefault42.9%±0.0not datedhtihle.github.io
101moonshotai/Kimi-K2-ThinkingEpoch: kimi-k2-thinking, Moonshotopendefault42.8%±0.0not datedhtihle.github.io
102grok-3-mini_highxAIclosedeffort high42.6%not datedWeirdML Leaderboard
103grok-3-mini-beta_highxAIclosedeffort high42.6%±0.0not datedWeirdML Leaderboard, via Epoch AI
104gemini-2.5-flash-preview-09-2025_16kGoogle DeepMindcloseddefault41.9%±0.0not datedhtihle.github.io
105openai/gpt-oss-120bEpoch: gpt-oss-120b_medium, OpenAIopeneffort medium41.9%±0.0not datedhtihle.github.io
106deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeekopendefault41.6%±0.0not datedhtihle.github.io
107Qwen/Qwen3-Coder-480B-A35B-InstructEpoch: Qwen3-Coder-480B-A35B-Instruct, Alibabaopendefault41.2%±0.0not datedhtihle.github.io
108Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault41.0%not datedWeirdML Leaderboard
109gemini-2.5-flash-preview-04-17 (16K thinking)Google DeepMindcloseddefault40.9%±0.0not datedWeirdML Leaderboard, via Epoch AI
110gemini-2.5-flash-preview-05-20_16KGoogle DeepMindcloseddefault40.9%not datedWeirdML Leaderboard
111openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAIopeneffort high40.9%±0.0not datedhtihle.github.io
112zai-org/GLM-4.5Epoch: glm-4.5_thinking, Z.ai (Zhipu AI),Tsinghua Universityopeneffort thinking40.6%±0.0not datedhtihle.github.io
113claude-3-5-sonnet-20241022Anthropiccloseddefault40.0%±0.0not datedhtihle.github.io
114gpt-5-chatOpenAIcloseddefault39.8%±0.0not datedhtihle.github.io
115Qwen/Qwen3.5-27BEpoch: qwen3.5-27B, Alibabaopendefault39.5%±0.0not datedhtihle.github.io
116deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_thinking, DeepSeekopeneffort thinking39.5%±0.0not datedhtihle.github.io
117gpt-4.5-preview-2025-02-27OpenAIcloseddefault39.4%±0.0not datedhtihle.github.io
118kimi-k2-0711-previewMoonshotno page matchopendefault39.4%±0.0not datedWeirdML Leaderboard, via Epoch AI
119moonshotai/Kimi-K2-InstructEpoch: Kimi-K2-Instruct, Moonshotopendefault39.4%not datedWeirdML Leaderboard
120gpt-4.1-2025-04-14OpenAIcloseddefault39.0%±0.0not datedhtihle.github.io
121gemini-3.5-flash-lite_highGoogle DeepMindclosedeffort high39.0%±0.0not datedhtihle.github.io
122Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: qwen/qwen3-235b-a22b-thinking-2507, Alibabaopendefault38.7%±0.0not datedWeirdML Leaderboard, via Epoch AI
123Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: Qwen3-235B-A22B-Instruct-2507, Alibabaopendefault38.7%not datedWeirdML Leaderboard
124deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1_thinking, DeepSeekopeneffort thinking38.4%±0.0not datedhtihle.github.io
125gpt-5-nano-2025-08-07_highOpenAIclosedeffort high38.1%±0.0not datedhtihle.github.io
126nemotron-3-superNvidiano page matchopendefault38.0%±0.0not datedhtihle.github.io
127gpt-5.4-nano-2026-03-17_noneOpenAIclosedeffort none38.0%±0.0not datedhtihle.github.io
128gpt-5.4-mini-2026-03-17_noneOpenAIclosedeffort none37.9%±0.0not datedhtihle.github.io
129gpt-4.1-mini-2025-04-14OpenAIcloseddefault37.6%±0.0not datedhtihle.github.io
130deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1, DeepSeekopendefault37.5%±0.0not datedhtihle.github.io
131Qwen/Qwen3-235B-A22BEpoch: Qwen3-235B-A22B, Alibabaopendefault37.3%±0.0not datedWeirdML Leaderboard, via Epoch AI
132Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault37.3%not datedWeirdML Leaderboard
133grok-3xAIcloseddefault37.2%±0.0not datedhtihle.github.io
134MiniMaxAI/MiniMax-M2.7Epoch: MiniMax-M2.7, MiniMaxopendefault37.0%±0.0not datedhtihle.github.io
135openai/gpt-oss-20bEpoch: gpt-oss-20b_medium, OpenAIopeneffort medium36.8%±0.0not datedhtihle.github.io
136moonshotai/Kimi-K2-Instruct-0905Epoch: Kimi-K2-Instruct-0905, Moonshotopendefault36.7%±0.0not datedhtihle.github.io
137deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeekopendefault36.5%±0.0not datedhtihle.github.io
138o1-mini-2024-09-12_mediumOpenAIclosedeffort medium36.3%±0.0not datedhtihle.github.io
139deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeekopendefault36.1%±0.0not datedhtihle.github.io
140gemini-2.5-flash-lite-preview-06-17_16KGoogle DeepMindcloseddefault35.2%not datedWeirdML Leaderboard
141gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMindcloseddefault35.2%±0.0not datedWeirdML Leaderboard, via Epoch AI
142gemma-4-26b-a4bGoogle DeepMindno page matchopendefault35.2%±0.0not datedhtihle.github.io
143grok-code-fast-1not stateddefault35.1%±0.0not datedhtihle.github.io
144Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibabaopendefault34.5%±0.0not datedhtihle.github.io
145Qwen/Qwen3-Coder-NextEpoch: qwen3-coder-nextopendefault34.4%±0.0not datedWeirdML Leaderboard, via Epoch AI
146mistral-medium-2508Mistral AIcloseddefault33.1%±0.0not datedhtihle.github.io
147thinkingmachines/InklingEpoch: Inkling_high, Thinking Machinesopeneffort high32.3%±0.0not datedhtihle.github.io
148claude-3-5-sonnet-20240620Anthropiccloseddefault31.0%±0.0not datedhtihle.github.io
149claude-3-5-haiku-20241022Anthropiccloseddefault30.7%±0.0not datedhtihle.github.io
150Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibabaopendefault29.8%not datedWeirdML Leaderboard
151gpt-5-nano-2025-08-07_lowOpenAIclosedeffort low25.9%±0.0not datedhtihle.github.io
152gemini-2.0-flash-001Google DeepMind,Googlecloseddefault25.8%±0.0not datedhtihle.github.io
153gpt-4o-2024-11-20OpenAIcloseddefault25.1%±0.0not datedhtihle.github.io
154gemini-1.5-flash-002Google DeepMindcloseddefault24.9%±0.0not datedhtihle.github.io
155meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AIopendefault24.5%±0.0not datedhtihle.github.io
156grok-2-1212xAIcloseddefault22.2%±0.0not datedhtihle.github.io
157gemini-1.5-pro-002Google DeepMindcloseddefault22.2%±0.0not datedhtihle.github.io
158meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AIopendefault21.4%±0.0not datedhtihle.github.io
159claude-3-opus-20240229Anthropiccloseddefault19.2%±0.0not datedhtihle.github.io
160gpt-4.1-nano-2025-04-14OpenAIcloseddefault19.0%±0.0not datedhtihle.github.io
161gpt-4-turbo-2024-04-09OpenAIcloseddefault18.0%±0.0not datedhtihle.github.io
162Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibabaopendefault16.0%±0.0not datedWeirdML Leaderboard, via Epoch AI
163meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopendefault14.4%±0.0not datedhtihle.github.io
164gpt-4-0613OpenAIcloseddefault12.4%±0.0not datedhtihle.github.io
165gpt-4o-mini-2024-07-18OpenAIcloseddefault11.8%±0.0not datedhtihle.github.io
166Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, Alibabaopendefault11.3%±0.0not datedhtihle.github.io
167claude-3-sonnet-20240229Anthropiccloseddefault10.2%±0.0not datedhtihle.github.io
168claude-3-haiku-20240307Anthropiccloseddefault9.8%±0.0not datedhtihle.github.io
169meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AIopendefault9.0%±0.0not datedhtihle.github.io
170claude-2.1Anthropiccloseddefault7.1%±0.0not datedhtihle.github.io
171gpt-3.5-turbo-0125OpenAIcloseddefault3.5%±0.0not datedhtihle.github.io
172mistralai/Mixtral-8x22B-Instruct-v0.1Epoch: Mixtral-8x22B-Instruct-v0.1, Mistral AIopendefault3.2%±0.0not datedhtihle.github.io
173meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopendefault1.7%±0.0not datedhtihle.github.io

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method