Skip to content
Weights Not Walls
Menu

SimpleQA Verified

Re-run by Epoch AI

No description yet: we only describe an eval set when we can do it correctly.

Kind
Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
Score shown
Epoch's column "Best score (across scorers)", as a percentage, higher is better; random guessing scores about 0.0%. Error bars as the source gives them.
Results
105 in all: 21 for open-weight models, 80 for closed ones, 4 where Epoch does not say. 19 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K3.1-Pro, 62.2%, number 19 of 105 results.

SimpleQA Verified results
#ModelWeightsSettingScoreDateSource
1claude-fable-5-2_maxAnthropicclosedeffort max94.0%±0.86 Nov 2026Epoch AI
2gpt-6.2-astra_maxOpenAIclosedeffort max78.5%±1.327 Oct 2026Epoch AI
3gemini-4-carbon_highGoogle DeepMindclosedeffort high77.7%±1.32 Nov 2026Epoch AI
4gemini-4-argon_highGoogle DeepMindclosedeffort high76.0%±1.413 Oct 2026Epoch AI log
5gpt-6-astra_maxOpenAIclosedeffort max75.6%±1.430 Aug 2026Epoch AI log
6gpt-6.1-sol_maxOpenAIclosedeffort max73.9%±1.429 Sep 2026Epoch AI log
7gemini-3.1-pro-preview_highGoogle DeepMindclosedeffort high73.5%±1.410 Aug 2026Epoch AI
8claude-opus-5-5_maxAnthropicclosedeffort max72.2%±1.422 Sep 2026Epoch AI log
9gemini-4-hydrogen_xhighGoogle DeepMindclosedeffort xhigh70.9%±1.413 Nov 2026Epoch AI log
10claude-fable-5-1_maxAnthropicclosedeffort max70.8%±1.41 Sep 2026Epoch AI log
11claude-fable-5_xhighAnthropicclosedeffort xhigh70.7%±1.410 Aug 2026Epoch AI
12gemini-3.8-flash_highGoogle DeepMindclosedeffort high69.7%±1.52 Sep 2026Epoch AI log
13gpt-5.6-sol_maxOpenAIclosedeffort max69.7%±1.510 Aug 2026Epoch AI
14gemini-3.7-flash_highGoogle DeepMindclosedeffort high69.2%±1.527 Aug 2026Epoch AI log
15gemini-3-flash-preview_highGoogle DeepMindclosedeffort high66.8%±1.527 Aug 2026Epoch AI log
16gemini-3.5-flash_highGoogle DeepMindclosedeffort high66.2%±1.527 Aug 2026Epoch AI log
17gemini-3.6-flash_highGoogle DeepMindclosedeffort high66.2%±1.527 Aug 2026Epoch AI log
18gpt-5.5_xhighOpenAIclosedeffort xhigh63.0%±1.527 Aug 2026Epoch AI log
19moonshotai/Kimi-K3.1-ProEpoch: kimi-k3.1-pro-together_max, Moonshotopeneffort max62.2%±1.56 Nov 2026Epoch AI log
20gpt-6-sol_maxOpenAIclosedeffort max60.7%±1.522 Sep 2026Epoch AI log
21muse-spark-1.2_xhighMeta AIclosedeffort xhigh60.3%±1.627 Aug 2026Epoch AI log
22claude-opus-5_maxAnthropicclosedeffort max59.9%±1.610 Aug 2026Epoch AI
23muse-spark-1.1Meta AIcloseddefault57.8%±1.631 Aug 2026Epoch AI log
24gpt-6.1-terra_maxOpenAIclosedeffort max57.3%±1.626 Oct 2026Epoch AI log
25meta-models/Muse-Spark-Open-816B-A42BEpoch: Muse-Spark-Open-816B-A42B_xhigh, Meta AIopeneffort xhigh56.6%±1.615 Oct 2026Epoch AI log
26grok-4.7_xhighxAIclosedeffort xhigh56.0%±1.622 Sep 2026Epoch AI log
27qwen3.7-maxAlibabacloseddefault55.8%±1.627 Aug 2026Epoch AI log
28claude-opus-4-8_maxAnthropicclosedeffort max53.0%±1.627 Aug 2026Epoch AI log
29deepseek-ai/DeepSeek-V4-Pro-0813Epoch: deepseek-v4-pro-0813_max, DeepSeekopeneffort max52.9%±1.627 Aug 2026Epoch AI log
30qwen3.6-max-previewAlibabanot stateddefault52.0%±1.627 Aug 2026Epoch AI log
31claude-opus-4-7_xhighAnthropicclosedeffort xhigh51.7%±1.627 Aug 2026Epoch AI log
32moonshotai/Kimi-K3Epoch: kimi-k3_max, Moonshotopeneffort max50.6%±1.627 Aug 2026Epoch AI log
33gpt-5-2025-08-07_highOpenAIclosedeffort high50.1%±1.627 Aug 2026Epoch AI log
34o3-2025-04-16_highOpenAIclosedeffort high49.4%±1.627 Aug 2026Epoch AI log
35grok-4.6_highxAIclosedeffort high49.3%±1.627 Aug 2026Epoch AI log
36grok-4.6_xhighxAIclosedeffort xhigh48.9%±1.627 Aug 2026Epoch AI log
37qwen3-max-2025-09-23Alibabacloseddefault48.7%±1.627 Aug 2026Epoch AI log
38grok-4.5_highxAIclosedeffort high48.3%±1.627 Aug 2026Epoch AI log
39gpt-5.1-2025-11-13_highOpenAIclosedeffort high48.0%±1.627 Aug 2026Epoch AI log
40qwen3.8-max-0902_xhighAlibabaclosedeffort xhigh47.3%±1.62 Sep 2026Epoch AI log
41claude-opus-4-6_maxAnthropicclosedeffort max47.0%±1.627 Aug 2026Epoch AI log
42deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeekopeneffort max47.0%±1.627 Aug 2026Epoch AI log
43claude-sonnet-5-5_maxAnthropicclosedeffort max46.5%±1.629 Sep 2026Epoch AI log
44gpt-5.4-pro-2026-03-05_xhighOpenAIclosedeffort xhigh46.3%±1.627 Aug 2026Epoch AI log
45qwen3.8-max_xhighAlibabaclosedeffort xhigh45.8%±1.627 Aug 2026Epoch AI log
46claude-opus-4-5-20251101_32KAnthropiccloseddefault45.7%±1.627 Aug 2026Epoch AI log
47gpt-5.4-2026-03-05_xhighOpenAIclosedeffort xhigh45.1%±1.627 Aug 2026Epoch AI log
48doubao-seed-2.1-pro-260628_highByteDanceclosedeffort high44.7%±1.628 Oct 2026Epoch AI log
49qwen3.6-plusAlibabacloseddefault44.1%±1.627 Aug 2026Epoch AI log
50gpt-5.6-terra_maxOpenAIclosedeffort max43.2%±1.610 Aug 2026Epoch AI
51deepseek-ai/DeepSeek-V4.1-FlashEpoch: deepseek-v4.1-flash_max, DeepSeekopeneffort max42.9%±1.613 Oct 2026Epoch AI log
52deepseek-ai/DeepSeek-V4.1-Flash-1017Epoch: deepseek-v4.1-flash-1017_max, DeepSeekopeneffort max42.5%±1.619 Oct 2026Epoch AI log
53doubao-seed-evolving_highByteDanceclosedeffort high41.9%±1.62 Nov 2026Epoch AI log
54gpt-6-luna_maxOpenAIclosedeffort max41.4%±1.622 Sep 2026Epoch AI log
55o1-2024-12-17_highOpenAIclosedeffort high41.1%±1.631 Aug 2026Epoch AI log
56zai-org/GLM-5.3Epoch: glm-5.3_max, Z.ai (Zhipu AI)openeffort max41.0%±1.628 Aug 2026Epoch AI log
57gpt-5.6-luna_maxOpenAIclosedeffort max41.0%±1.610 Aug 2026Epoch AI
58Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault40.4%±1.627 Aug 2026Epoch AI log
59thinkingmachines/InklingEpoch: Inkling_xhigh, Thinking Machinesopeneffort xhigh40.3%±1.627 Aug 2026Epoch AI log
60doubao-seed-2.0-pro-260215_highByteDanceclosedeffort high39.4%±1.528 Oct 2026Epoch AI log
61gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh37.1%±1.527 Aug 2026Epoch AI log
62moonshotai/Kimi-K2.7-CodeEpoch: kimi-k2.7-code, Moonshotopendefault36.5%±1.527 Aug 2026Epoch AI log
63claude-sonnet-4-6_highAnthropicclosedeffort high35.5%±1.510 Aug 2026Epoch AI
64moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault34.9%±1.510 Aug 2026Epoch AI
65gpt-5.2-2025-12-11_highOpenAIclosedeffort high34.3%±1.527 Aug 2026Epoch AI log
66moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshotopendefault34.3%±1.527 Aug 2026Epoch AI log
67zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI)openeffort max34.2%±1.527 Aug 2026Epoch AI log
68zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI)opendefault34.0%±1.527 Aug 2026Epoch AI log
69claude-sonnet-5_maxAnthropicclosedeffort max33.7%±1.527 Aug 2026Epoch AI log
70deepseek-ai/DeepSeek-V4-Flash-0731Epoch: deepseek-v4-flash-0731_max, DeepSeekopeneffort max33.6%±1.527 Aug 2026Epoch AI log
71grok-4.3_highxAIclosedeffort high33.2%±1.527 Aug 2026Epoch AI log
72claude-sonnet-5_xhighAnthropicclosedeffort xhigh32.9%±1.527 Aug 2026Epoch AI log
73claude-sonnet-4-6_maxAnthropicclosedeffort max32.8%±1.527 Aug 2026Epoch AI log
74gpt-5.2-2025-12-11_lowOpenAIclosedeffort low32.8%±1.527 Aug 2026Epoch AI log
75gpt-5.2-2025-12-11_mediumOpenAIclosedeffort medium32.7%±1.527 Aug 2026Epoch AI log
76zai-org/GLM-4.7Epoch: glm-4.7, Z.ai (Zhipu AI)opendefault32.2%±1.527 Aug 2026Epoch AI log
77doubao-seed-2.0-lite-260428_highByteDanceclosedeffort high31.5%±1.528 Oct 2026Epoch AI log
78gpt-4.1-2025-04-14OpenAIcloseddefault31.1%±1.531 Aug 2026Epoch AI log
79claude-sonnet-4-5-20250929_59KAnthropiccloseddefault30.7%±1.527 Aug 2026Epoch AI log
80grok-4.20-0309-reasoningxAIcloseddefault30.2%±1.527 Aug 2026Epoch AI log
81doubao-seed-2.1-turbo-260628_highByteDanceclosedeffort high29.7%±1.430 Oct 2026Epoch AI log
82gpt-5.4-mini-2026-03-17_highOpenAIclosedeffort high29.4%±1.427 Aug 2026Epoch AI log
83gpt-4o-2024-08-06OpenAIcloseddefault26.0%±1.431 Aug 2026Epoch AI log
84qwen3.5-plusAlibabanot stateddefault25.4%±1.427 Aug 2026Epoch AI log
85claude-haiku-5-5_maxAnthropicclosedeffort max23.8%±1.39 Oct 2026Epoch AI log
86claude-sonnet-4-5-20250929Anthropiccloseddefault23.7%±1.310 Aug 2026Epoch AI
87mistral-large-4-0-oss_highMistral AIno page matchopeneffort high22.4%±1.32 Nov 2026Epoch AI log
88gpt-5-mini-2025-08-07_highOpenAIclosedeffort high21.6%±1.310 Aug 2026Epoch AI
89qwen3.5-flashAlibabanot stateddefault20.3%±1.327 Aug 2026Epoch AI log
90mistral-large-4-0_highMistral AIno page matchopeneffort high20.0%±1.39 Oct 2026Epoch AI log
91o4-mini-2025-04-16_lowOpenAIclosedeffort low19.6%±1.327 Aug 2026Epoch AI log
92thinkingmachines/Inkling-SmallEpoch: inkling-small_xhigh, Thinking Machinesopeneffort xhigh19.1%±1.227 Aug 2026Epoch AI log
93o4-mini-2025-04-16_highOpenAIclosedeffort high18.8%±1.227 Aug 2026Epoch AI log
94seed-1.6ByteDancecloseddefault17.4%±1.230 Oct 2026Epoch AI log
95qwen3.6-flashAlibabanot stateddefault15.9%±1.227 Aug 2026Epoch AI log
96o3-mini-2025-01-31_highOpenAIclosedeffort high15.3%±1.131 Aug 2026Epoch AI log
97claude-haiku-4-5-20251001Anthropiccloseddefault13.2%±1.110 Aug 2026Epoch AI
98gpt-4.1-mini-2025-04-14OpenAIcloseddefault12.7%±1.131 Aug 2026Epoch AI log
99claude-3-opus-20240229Anthropiccloseddefault12.6%±1.031 Aug 2026Epoch AI log
100claude-haiku-4-5-20251001_32KAnthropiccloseddefault12.6%±1.027 Aug 2026Epoch AI log
101gpt-5-nano-2025-08-07_highOpenAIclosedeffort high11.7%±1.010 Aug 2026Epoch AI
102gpt-5.4-nano-2026-03-17_highOpenAIclosedeffort high11.7%±1.027 Aug 2026Epoch AI log
103google/gemma-4-31B-itEpoch: gemma-4-31b-it, Google DeepMindopendefault10.4%±1.027 Aug 2026Epoch AI log
104gpt-4o-mini-2024-07-18OpenAIcloseddefault8.3%±0.931 Aug 2026Epoch AI log
105gpt-4.1-nano-2025-04-14OpenAIcloseddefault6.0%±0.831 Aug 2026Epoch AI log

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method