Skip to content
Weights Not Walls
Menu

CadEval

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://willpatrick.xyz/cadevalresults_20250422_095709/
Score shown
Epoch's column "Overall pass (%)", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
Results
15 in all: 0 for open-weight models, 14 for closed ones, 1 where Epoch does not say. 0 linked to a model page here.

No open-weight model has a result on this set.

CadEval results
#ModelWeightsSettingScoreDateSource
1o3-2025-04-16_mediumOpenAIclosedeffort medium74.0%not datedCadEval Dashboard
2gemini-2.5-pro-preview-03-25Google DeepMindcloseddefault64.0%not datedCadEval Dashboard
3o4-mini-2025-04-16_mediumOpenAIclosedeffort medium62.0%not datedCadEval Dashboard
4o1-2024-12-17_mediumOpenAIclosedeffort medium56.0%not datedCadEval Dashboard
5claude-3-7-sonnet-20250219Anthropiccloseddefault54.0%not datedCadEval Dashboard
6o3-mini-2025-01-31_mediumOpenAIclosedeffort medium54.0%not datedCadEval Dashboard
7claude-3-5-sonnet-20241022Anthropiccloseddefault48.0%not datedCadEval Dashboard
8gpt-4.1-2025-04-14OpenAIcloseddefault42.0%not datedCadEval Dashboard
9gemini-1.5-pro-002Google DeepMindcloseddefault34.0%not datedCadEval Dashboard
10claude-3-5-haiku-20241022Anthropiccloseddefault32.0%not datedCadEval Dashboard
11gemini-2.0-flash-001Google DeepMind,Googlecloseddefault30.0%not datedCadEval Dashboard
12gpt-4o-2024-08-06OpenAIcloseddefault26.0%not datedCadEval Dashboard
13ml-elephantnot stateddefault20.0%not datedCadEval Dashboard
14gpt-4.1-mini-2025-04-14OpenAIcloseddefault16.0%not datedCadEval Dashboard
15claude-3-haiku-20240307Anthropiccloseddefault12.0%not datedCadEval Dashboard

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method