WeirdML
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://htihle.github.io/weirdml.html
- Score shown
- Epoch's column "Accuracy", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
- Results
- 173 in all: 58 for open-weight models, 114 for closed ones, 1 where Epoch does not say. 53 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K3, 82.6%, number 18 of 173 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | gpt-6-astra_promaxOpenAI | closed | default | 93.6%±0.0 | not dated | htihle.github.io |
| 2 | gpt-6-astra_maxOpenAI | closed | effort max | 93.3%±0.0 | not dated | htihle.github.io |
| 3 | claude-fable-5-1_maxAnthropic | closed | effort max | 92.9%±0.0 | not dated | htihle.github.io |
| 4 | gpt-6-astra_highOpenAI | closed | effort high | 92.9%±0.0 | not dated | htihle.github.io |
| 5 | claude-fable-5-1_highAnthropic | closed | effort high | 92.3%±0.0 | not dated | htihle.github.io |
| 6 | claude-fable-5_maxAnthropic | closed | effort max | 91.9%±0.0 | not dated | htihle.github.io |
| 7 | claude-opus-5_maxAnthropic | closed | effort max | 91.8%±0.0 | not dated | htihle.github.io |
| 8 | claude-opus-5_highAnthropic | closed | effort high | 91.6%±0.0 | not dated | htihle.github.io |
| 9 | gpt-5.6-sol_promaxOpenAI | closed | default | 89.4%±0.0 | not dated | htihle.github.io |
| 10 | gpt-5.6-sol_highOpenAI | closed | effort high | 88.8%±0.0 | not dated | htihle.github.io |
| 11 | claude-fable-5_highAnthropic | closed | effort high | 87.8%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 12 | gpt-5.6-sol_maxOpenAI | closed | effort max | 87.0%±0.0 | not dated | htihle.github.io |
| 13 | claude-opus-5Anthropic | closed | default | 86.3%±0.0 | not dated | htihle.github.io |
| 14 | gpt-5.5_xhighOpenAI | closed | effort xhigh | 84.9%±0.0 | not dated | htihle.github.io |
| 15 | gemini-3.8-flash_highGoogle DeepMind | closed | effort high | 84.8%±0.0 | not dated | htihle.github.io |
| 16 | gpt-5.5_highOpenAI | closed | effort high | 83.9%±0.0 | not dated | htihle.github.io |
| 17 | claude-opus-4-8_xhighAnthropic | closed | effort xhigh | 82.9%±0.0 | not dated | htihle.github.io |
| 18 | moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshot | open | effort max | 82.6%±0.0 | not dated | htihle.github.io |
| 19 | gpt-5.3-codexOpenAI | closed | default | 79.3% | not dated | WeirdML Leaderboard, via Epoch AI |
| 20 | gpt-5.6-terra_highOpenAI | closed | effort high | 78.3%±0.0 | not dated | htihle.github.io |
| 21 | claude-opus-4-6_highAnthropic | closed | effort high | 78.0%±0.0 | not dated | htihle.github.io |
| 22 | claude-opus-4-6_unknownAnthropic | closed | effort unknown | 77.9% | not dated | WeirdML Leaderboard, via Epoch AI |
| 23 | gpt-5.3-codex_xhighOpenAI | closed | effort xhigh | 77.9%±0.0 | not dated | htihle.github.io |
| 24 | gpt-5.4-2026-03-05_xhighOpenAI | closed | effort xhigh | 77.7%±0.0 | not dated | htihle.github.io |
| 25 | claude-opus-4-7_highAnthropic | closed | effort high | 76.4%±0.0 | not dated | htihle.github.io |
| 26 | claude-opus-4-7Anthropic | closed | default | 76.4%±0.0 | not dated | htihle.github.io |
| 27 | claude-opus-4-7_unknownAnthropic | closed | effort unknown | 76.4% | not dated | WeirdML Leaderboard, via Epoch AI |
| 28 | claude-opus-4-8_mediumAnthropic | closed | effort medium | 76.0%±0.0 | not dated | htihle.github.io |
| 29 | claude-opus-4-7_maxAnthropic | closed | effort max | 75.5%±0.0 | not dated | htihle.github.io |
| 30 | zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI) | open | effort max | 75.4%±0.0 | not dated | htihle.github.io |
| 31 | gpt-5.2-2025-12-11_xhighOpenAI | closed | effort xhigh | 72.2%±0.0 | not dated | htihle.github.io |
| 32 | gemini-3.1-pro-previewGoogle DeepMind | closed | default | 72.1%±0.0 | not dated | htihle.github.io |
| 33 | claude-opus-4-8_noneAnthropic | closed | effort none | 70.5%±0.0 | not dated | htihle.github.io |
| 34 | zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI) | open | effort max | 70.1%±0.0 | not dated | htihle.github.io |
| 35 | gemini-3-pro-previewGoogle DeepMind | closed | default | 69.9%±0.0 | not dated | htihle.github.io |
| 36 | claude-sonnet-5_highAnthropic | closed | effort high | 69.0%±0.0 | not dated | htihle.github.io |
| 37 | zai-org/GLM-5.2Epoch: glm-5.2_high, Z.ai (Zhipu AI) | open | effort high | 67.3%±0.0 | not dated | htihle.github.io |
| 38 | grok-4.6_highxAI | closed | effort high | 67.3%±0.0 | not dated | htihle.github.io |
| 39 | gpt-5.5_noneOpenAI | closed | effort none | 67.2%±0.0 | not dated | htihle.github.io |
| 40 | deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_max, DeepSeek | open | effort max | 66.2%±0.0 | not dated | htihle.github.io |
| 41 | claude-sonnet-4-6_mediumAnthropic | closed | effort medium | 66.1%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 42 | claude-opus-4-6Anthropic | closed | default | 65.9%±0.0 | not dated | htihle.github.io |
| 43 | claude-opus-4-5-20251101_16KAnthropic | closed | default | 63.7%±0.0 | not dated | htihle.github.io |
| 44 | gpt-5.2-2025-12-11_mediumOpenAI | closed | effort medium | 63.4%±0.0 | not dated | htihle.github.io |
| 45 | deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeek | open | effort max | 63.0%±0.0 | not dated | htihle.github.io |
| 46 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 62.6%±0.0 | not dated | htihle.github.io |
| 47 | gemini-3-flash-previewGoogle DeepMind | closed | default | 61.6%±0.0 | not dated | htihle.github.io |
| 48 | gpt-5.6-luna_highOpenAI | closed | effort high | 60.9%±0.0 | not dated | htihle.github.io |
| 49 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 60.8%±0.0 | not dated | htihle.github.io |
| 50 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 60.7%±0.0 | not dated | htihle.github.io |
| 51 | gpt-5-pro-2025-10-06_highOpenAI | closed | effort high | 60.4%±0.0 | not dated | htihle.github.io |
| 52 | muse-spark-1.2_xhighMeta AI | closed | effort xhigh | 60.4%±0.0 | not dated | htihle.github.io |
| 53 | gpt-5.4-mini-2026-03-17_highOpenAI | closed | effort high | 60.3%±0.0 | not dated | htihle.github.io |
| 54 | o3-pro-2025-06-10_highOpenAI | closed | effort high | 58.2%±0.0 | not dated | htihle.github.io |
| 55 | gpt-5.4-2026-03-05_noneOpenAI | closed | effort none | 57.4%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 56 | gpt-5.4-pro-2026-03-05_noneOpenAI | closed | effort none | 57.4%±0.0 | not dated | htihle.github.io |
| 57 | zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI) | open | default | 57.1%±0.0 | not dated | htihle.github.io |
| 58 | deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_high, DeepSeek | open | effort high | 57.0%±0.0 | not dated | htihle.github.io |
| 59 | gemini-3.6-flash_highGoogle DeepMind | closed | effort high | 56.1%±0.0 | not dated | htihle.github.io |
| 60 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 55.9%±0.0 | not dated | htihle.github.io |
| 61 | gpt-5-codexOpenAI | closed | default | 54.5% | not dated | WeirdML Leaderboard |
| 62 | gpt-5-codex_highOpenAI | closed | effort high | 54.5%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 63 | moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, Moonshot | open | default | 54.1%±0.0 | not dated | htihle.github.io |
| 64 | gemini-2.5-pro_16KGoogle DeepMind | closed | default | 54.0% | not dated | WeirdML Leaderboard |
| 65 | gpt-5-mini-2025-08-07_highOpenAI | closed | effort high | 52.7%±0.0 | not dated | htihle.github.io |
| 66 | o4-mini-2025-04-16_highOpenAI | closed | effort high | 52.6%±0.0 | not dated | htihle.github.io |
| 67 | o3-2025-04-16_highOpenAI | closed | effort high | 52.4%±0.0 | not dated | htihle.github.io |
| 68 | google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMind | open | default | 52.3%±0.0 | not dated | htihle.github.io |
| 69 | grok-4-20xAI | closed | default | 52.3%±0.0 | not dated | htihle.github.io |
| 70 | gemini-3.1-flash-liteGoogle | closed | default | 52.2%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 71 | grok-4.3_unknownxAI | closed | effort unknown | 49.9%±0.0 | not dated | htihle.github.io |
| 72 | gpt-5.2-2025-12-11_lowOpenAI | closed | effort low | 49.6% | not dated | WeirdML Leaderboard |
| 73 | gpt-5.2-2025-12-11_noneOpenAI | closed | effort none | 49.6%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 74 | gpt-5.4-nano-2026-03-17_highOpenAI | closed | effort high | 49.2%±0.0 | not dated | htihle.github.io |
| 75 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeek | open | effort max | 48.6%±0.0 | not dated | htihle.github.io |
| 76 | zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI) | open | default | 48.2%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 77 | openai/gpt-oss-120bEpoch: gpt-oss-120b_high, OpenAI | open | effort high | 48.2% | not dated | WeirdML Leaderboard |
| 78 | openai/gpt-oss-120bEpoch: openai/gpt-oss-120b_high, OpenAI | open | effort high | 48.2%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 79 | claude-sonnet-4-5-20250929_16KAnthropic | closed | default | 47.7%±0.0 | not dated | htihle.github.io |
| 80 | o1-preview-2024-09-12OpenAI | closed | default | 47.6%±0.0 | not dated | htihle.github.io |
| 81 | deepseek-ai/DeepSeek-V3.2-SpecialeEpoch: DeepSeek-V3.2-Speciale, DeepSeek | open | default | 46.7%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 82 | claude-sonnet-4-5-20250929Anthropic | closed | default | 46.7%±0.0 | not dated | htihle.github.io |
| 83 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_high, DeepSeek | open | effort high | 46.5%±0.0 | not dated | htihle.github.io |
| 84 | grok-4.5_unknownxAI | closed | effort unknown | 46.4%±0.0 | not dated | htihle.github.io |
| 85 | claude-sonnet-4-20250514_16KAnthropic | closed | default | 46.1%±0.0 | not dated | htihle.github.io |
| 86 | o1-2024-12-17_highOpenAI | closed | effort high | 46.1%±0.0 | not dated | htihle.github.io |
| 87 | claude-opus-4-1-20250805_16KAnthropic | closed | default | 45.9%±0.0 | not dated | htihle.github.io |
| 88 | grok-4-0709xAI | closed | default | 45.7%±0.0 | not dated | htihle.github.io |
| 89 | deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_max, DeepSeek | open | effort max | 45.6%±0.0 | not dated | htihle.github.io |
| 90 | moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshot | open | default | 45.6%±0.0 | not dated | htihle.github.io |
| 91 | claude-haiku-4-5-20251001Anthropic | closed | default | 45.4%±0.0 | not dated | htihle.github.io |
| 92 | claude-haiku-4-5-20251001_16KAnthropic | closed | default | 44.1%±0.0 | not dated | htihle.github.io |
| 93 | claude-sonnet-4-20250514Anthropic | closed | default | 43.9%±0.0 | not dated | htihle.github.io |
| 94 | deepseek-ai/DeepSeek-V4-FlashEpoch: deepseek-v4-flash_high, DeepSeek | open | effort high | 43.8%±0.0 | not dated | htihle.github.io |
| 95 | o3-mini-2025-01-31_highOpenAI | closed | effort high | 43.8%±0.0 | not dated | htihle.github.io |
| 96 | claude-opus-4-20250514_16KAnthropic | closed | default | 43.7%±0.0 | not dated | htihle.github.io |
| 97 | mistral-medium-2604Mistral AIno page match | open | default | 43.7%±0.0 | not dated | htihle.github.io |
| 98 | nemotron-3-ultraNvidiano page match | open | default | 43.5%±0.0 | not dated | htihle.github.io |
| 99 | mercury-2Inception Labs | closed | default | 43.2%±0.0 | not dated | htihle.github.io |
| 100 | grok-4-fastxAI | closed | default | 42.9%±0.0 | not dated | htihle.github.io |
| 101 | moonshotai/Kimi-K2-ThinkingEpoch: kimi-k2-thinking, Moonshot | open | default | 42.8%±0.0 | not dated | htihle.github.io |
| 102 | grok-3-mini_highxAI | closed | effort high | 42.6% | not dated | WeirdML Leaderboard |
| 103 | grok-3-mini-beta_highxAI | closed | effort high | 42.6%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 104 | gemini-2.5-flash-preview-09-2025_16kGoogle DeepMind | closed | default | 41.9%±0.0 | not dated | htihle.github.io |
| 105 | openai/gpt-oss-120bEpoch: gpt-oss-120b_medium, OpenAI | open | effort medium | 41.9%±0.0 | not dated | htihle.github.io |
| 106 | deepseek-ai/DeepSeek-R1-0528Epoch: DeepSeek-R1-0528, DeepSeek | open | default | 41.6%±0.0 | not dated | htihle.github.io |
| 107 | Qwen/Qwen3-Coder-480B-A35B-InstructEpoch: Qwen3-Coder-480B-A35B-Instruct, Alibaba | open | default | 41.2%±0.0 | not dated | htihle.github.io |
| 108 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 41.0% | not dated | WeirdML Leaderboard |
| 109 | gemini-2.5-flash-preview-04-17 (16K thinking)Google DeepMind | closed | default | 40.9%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 110 | gemini-2.5-flash-preview-05-20_16KGoogle DeepMind | closed | default | 40.9% | not dated | WeirdML Leaderboard |
| 111 | openai/gpt-oss-20bEpoch: gpt-oss-20b_high, OpenAI | open | effort high | 40.9%±0.0 | not dated | htihle.github.io |
| 112 | zai-org/GLM-4.5Epoch: glm-4.5_thinking, Z.ai (Zhipu AI),Tsinghua University | open | effort thinking | 40.6%±0.0 | not dated | htihle.github.io |
| 113 | claude-3-5-sonnet-20241022Anthropic | closed | default | 40.0%±0.0 | not dated | htihle.github.io |
| 114 | gpt-5-chatOpenAI | closed | default | 39.8%±0.0 | not dated | htihle.github.io |
| 115 | Qwen/Qwen3.5-27BEpoch: qwen3.5-27B, Alibaba | open | default | 39.5%±0.0 | not dated | htihle.github.io |
| 116 | deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_thinking, DeepSeek | open | effort thinking | 39.5%±0.0 | not dated | htihle.github.io |
| 117 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 39.4%±0.0 | not dated | htihle.github.io |
| 118 | kimi-k2-0711-previewMoonshotno page match | open | default | 39.4%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 119 | moonshotai/Kimi-K2-InstructEpoch: Kimi-K2-Instruct, Moonshot | open | default | 39.4% | not dated | WeirdML Leaderboard |
| 120 | gpt-4.1-2025-04-14OpenAI | closed | default | 39.0%±0.0 | not dated | htihle.github.io |
| 121 | gemini-3.5-flash-lite_highGoogle DeepMind | closed | effort high | 39.0%±0.0 | not dated | htihle.github.io |
| 122 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: qwen/qwen3-235b-a22b-thinking-2507, Alibaba | open | default | 38.7%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 123 | Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: Qwen3-235B-A22B-Instruct-2507, Alibaba | open | default | 38.7% | not dated | WeirdML Leaderboard |
| 124 | deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1_thinking, DeepSeek | open | effort thinking | 38.4%±0.0 | not dated | htihle.github.io |
| 125 | gpt-5-nano-2025-08-07_highOpenAI | closed | effort high | 38.1%±0.0 | not dated | htihle.github.io |
| 126 | nemotron-3-superNvidiano page match | open | default | 38.0%±0.0 | not dated | htihle.github.io |
| 127 | gpt-5.4-nano-2026-03-17_noneOpenAI | closed | effort none | 38.0%±0.0 | not dated | htihle.github.io |
| 128 | gpt-5.4-mini-2026-03-17_noneOpenAI | closed | effort none | 37.9%±0.0 | not dated | htihle.github.io |
| 129 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 37.6%±0.0 | not dated | htihle.github.io |
| 130 | deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1, DeepSeek | open | default | 37.5%±0.0 | not dated | htihle.github.io |
| 131 | Qwen/Qwen3-235B-A22BEpoch: Qwen3-235B-A22B, Alibaba | open | default | 37.3%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 132 | Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibaba | open | default | 37.3% | not dated | WeirdML Leaderboard |
| 133 | grok-3xAI | closed | default | 37.2%±0.0 | not dated | htihle.github.io |
| 134 | MiniMaxAI/MiniMax-M2.7Epoch: MiniMax-M2.7, MiniMax | open | default | 37.0%±0.0 | not dated | htihle.github.io |
| 135 | openai/gpt-oss-20bEpoch: gpt-oss-20b_medium, OpenAI | open | effort medium | 36.8%±0.0 | not dated | htihle.github.io |
| 136 | moonshotai/Kimi-K2-Instruct-0905Epoch: Kimi-K2-Instruct-0905, Moonshot | open | default | 36.7%±0.0 | not dated | htihle.github.io |
| 137 | deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeek | open | default | 36.5%±0.0 | not dated | htihle.github.io |
| 138 | o1-mini-2024-09-12_mediumOpenAI | closed | effort medium | 36.3%±0.0 | not dated | htihle.github.io |
| 139 | deepseek-ai/DeepSeek-V3-0324Epoch: DeepSeek-V3-0324, DeepSeek | open | default | 36.1%±0.0 | not dated | htihle.github.io |
| 140 | gemini-2.5-flash-lite-preview-06-17_16KGoogle DeepMind | closed | default | 35.2% | not dated | WeirdML Leaderboard |
| 141 | gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMind | closed | default | 35.2%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 142 | gemma-4-26b-a4bGoogle DeepMindno page match | open | default | 35.2%±0.0 | not dated | htihle.github.io |
| 143 | grok-code-fast-1 | not stated | default | 35.1%±0.0 | not dated | htihle.github.io |
| 144 | Qwen/Qwen3.6-35B-A3BEpoch: qwen3.6-35b-a3b, Alibaba | open | default | 34.5%±0.0 | not dated | htihle.github.io |
| 145 | Qwen/Qwen3-Coder-NextEpoch: qwen3-coder-next | open | default | 34.4%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 146 | mistral-medium-2508Mistral AI | closed | default | 33.1%±0.0 | not dated | htihle.github.io |
| 147 | thinkingmachines/InklingEpoch: Inkling_high, Thinking Machines | open | effort high | 32.3%±0.0 | not dated | htihle.github.io |
| 148 | claude-3-5-sonnet-20240620Anthropic | closed | default | 31.0%±0.0 | not dated | htihle.github.io |
| 149 | claude-3-5-haiku-20241022Anthropic | closed | default | 30.7%±0.0 | not dated | htihle.github.io |
| 150 | Qwen/Qwen3-30B-A3BEpoch: Qwen3-30B-A3B, Alibaba | open | default | 29.8% | not dated | WeirdML Leaderboard |
| 151 | gpt-5-nano-2025-08-07_lowOpenAI | closed | effort low | 25.9%±0.0 | not dated | htihle.github.io |
| 152 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 25.8%±0.0 | not dated | htihle.github.io |
| 153 | gpt-4o-2024-11-20OpenAI | closed | default | 25.1%±0.0 | not dated | htihle.github.io |
| 154 | gemini-1.5-flash-002Google DeepMind | closed | default | 24.9%±0.0 | not dated | htihle.github.io |
| 155 | meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AI | open | default | 24.5%±0.0 | not dated | htihle.github.io |
| 156 | grok-2-1212xAI | closed | default | 22.2%±0.0 | not dated | htihle.github.io |
| 157 | gemini-1.5-pro-002Google DeepMind | closed | default | 22.2%±0.0 | not dated | htihle.github.io |
| 158 | meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AI | open | default | 21.4%±0.0 | not dated | htihle.github.io |
| 159 | claude-3-opus-20240229Anthropic | closed | default | 19.2%±0.0 | not dated | htihle.github.io |
| 160 | gpt-4.1-nano-2025-04-14OpenAI | closed | default | 19.0%±0.0 | not dated | htihle.github.io |
| 161 | gpt-4-turbo-2024-04-09OpenAI | closed | default | 18.0%±0.0 | not dated | htihle.github.io |
| 162 | Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, Alibaba | open | default | 16.0%±0.0 | not dated | WeirdML Leaderboard, via Epoch AI |
| 163 | meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AI | open | default | 14.4%±0.0 | not dated | htihle.github.io |
| 164 | gpt-4-0613OpenAI | closed | default | 12.4%±0.0 | not dated | htihle.github.io |
| 165 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 11.8%±0.0 | not dated | htihle.github.io |
| 166 | Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, Alibaba | open | default | 11.3%±0.0 | not dated | htihle.github.io |
| 167 | claude-3-sonnet-20240229Anthropic | closed | default | 10.2%±0.0 | not dated | htihle.github.io |
| 168 | claude-3-haiku-20240307Anthropic | closed | default | 9.8%±0.0 | not dated | htihle.github.io |
| 169 | meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AI | open | default | 9.0%±0.0 | not dated | htihle.github.io |
| 170 | claude-2.1Anthropic | closed | default | 7.1%±0.0 | not dated | htihle.github.io |
| 171 | gpt-3.5-turbo-0125OpenAI | closed | default | 3.5%±0.0 | not dated | htihle.github.io |
| 172 | mistralai/Mixtral-8x22B-Instruct-v0.1Epoch: Mixtral-8x22B-Instruct-v0.1, Mistral AI | open | default | 3.2%±0.0 | not dated | htihle.github.io |
| 173 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | default | 1.7%±0.0 | not dated | htihle.github.io |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method