Skip to content
Weights Not Walls
Menu

VPCT

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://cbrower.dev/vpct
Score shown
Epoch's column "Correct", as a percentage, higher is better; random guessing scores about 33.3%. The source gives no error bars.
Results
38 in all: 0 for open-weight models, 37 for closed ones, 1 where Epoch does not say. 0 linked to a model page here.

No open-weight model has a result on this set.

VPCT results
#ModelWeightsSettingScoreDateSource
1gemini-3-pro-previewGoogle DeepMindcloseddefault91.0%not datedVPCT leaderboard
2gpt-5.2-2025-12-11_xhighOpenAIclosedeffort xhigh84.0%not datedVPCT leaderboard
3gemini-3-flash-previewGoogle DeepMindcloseddefault72.6%not datedVPCT leaderboard
4gpt-5.2-2025-12-11_highOpenAIclosedeffort high67.0%not datedVPCT leaderboard
5gpt-5-2025-08-07_highOpenAIclosedeffort high66.0%not datedVPCT leaderboard
6gpt-5-2025-08-07_mediumOpenAIclosedeffort medium63.2%not datedVPCT leaderboard
7gpt-5.1-2025-11-13_highOpenAIclosedeffort high58.7%not datedVPCT leaderboard
8o4-mini-2025-04-16_mediumOpenAIclosedeffort medium57.5%not datedVPCT leaderboard
9gpt-5.1-2025-11-13_mediumOpenAIclosedeffort medium53.3%not datedVPCT leaderboard
10o3-2025-04-16_mediumOpenAIclosedeffort medium52.0%not datedVPCT leaderboard
11gemini-2.5-pro-preview-03-25Google DeepMindcloseddefault48.0%not datedVPCT leaderboard
12gemini-2.5-pro-preview-06-05Google DeepMindcloseddefault46.4%not datedVPCT leaderboard
13gemini-2.5-flash-preview-09-2025Google DeepMindcloseddefault46.2%not datedVPCT leaderboard
14gpt-4.5-preview-2025-02-27OpenAIcloseddefault45.0%not datedVPCT leaderboard
15gemini-robotics-er-1.5-previewnot stateddefault40.8%not datedVPCT leaderboard
16gemini-2.5-pro-preview-05-06Google DeepMindcloseddefault40.5%not datedVPCT leaderboard
17gpt-5-mini-2025-08-07_mediumOpenAIclosedeffort medium40.2%not datedVPCT leaderboard
18claude-opus-4-5-20251101_32KAnthropiccloseddefault40.0%not datedTweet from VPCT creator
19gpt-4o-2024-11-20OpenAIcloseddefault40.0%not datedVPCT leaderboard
20claude-sonnet-4-5-20250929_32KAnthropiccloseddefault39.8%not datedVPCT leaderboard
21claude-3-7-sonnet-20250219Anthropiccloseddefault39.0%not datedVPCT leaderboard
22gpt-5-mini-2025-08-07_highOpenAIclosedeffort high39.0%not datedVPCT leaderboard
23claude-opus-4-20250514_16KAnthropiccloseddefault38.0%not datedVPCT leaderboard
24claude-sonnet-4-5-20250929Anthropiccloseddefault38.0%not datedVPCT leaderboard
25gemini-2.5-flash-preview-04-17Google DeepMindcloseddefault38.0%not datedVPCT leaderboard
26gpt-5-nano-2025-08-07_highOpenAIclosedeffort high37.2%not datedVPCT leaderboard
27o1-2024-12-17_mediumOpenAIclosedeffort medium37.0%not datedVPCT leaderboard
28gpt-5-nano-2025-08-07_mediumOpenAIclosedeffort medium35.4%not datedVPCT leaderboard
29claude-3-7-sonnet-20250219_64KAnthropiccloseddefault35.0%not datedVPCT leaderboard
30claude-opus-4-1-20250805Anthropiccloseddefault35.0%not datedVPCT leaderboard
31claude-sonnet-4-20250514_32KAnthropiccloseddefault34.0%not datedVPCT leaderboard
32gpt-4o-mini-2024-07-18OpenAIcloseddefault34.0%not datedVPCT leaderboard
33claude-3-5-sonnet-20240620Anthropiccloseddefault33.0%not datedVPCT leaderboard
34claude-3-5-sonnet-20241022Anthropiccloseddefault33.0%not datedVPCT leaderboard
35claude-opus-4-1-20250805_16KAnthropiccloseddefault33.0%not datedVPCT leaderboard
36claude-opus-4-20250514Anthropiccloseddefault33.0%not datedVPCT leaderboard
37claude-sonnet-4-20250514Anthropiccloseddefault30.0%not datedVPCT leaderboard
38gemini-2.5-flash-lite-preview-09-2025Google DeepMindcloseddefault30.0%not datedVPCT leaderboard

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method