CadEval
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://willpatrick.xyz/cadevalresults_20250422_095709/
- Score shown
- Epoch's column "Overall pass (%)", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
- Results
- 15 in all: 0 for open-weight models, 14 for closed ones, 1 where Epoch does not say. 0 linked to a model page here.
No open-weight model has a result on this set.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 74.0% | not dated | CadEval Dashboard |
| 2 | gemini-2.5-pro-preview-03-25Google DeepMind | closed | default | 64.0% | not dated | CadEval Dashboard |
| 3 | o4-mini-2025-04-16_mediumOpenAI | closed | effort medium | 62.0% | not dated | CadEval Dashboard |
| 4 | o1-2024-12-17_mediumOpenAI | closed | effort medium | 56.0% | not dated | CadEval Dashboard |
| 5 | claude-3-7-sonnet-20250219Anthropic | closed | default | 54.0% | not dated | CadEval Dashboard |
| 6 | o3-mini-2025-01-31_mediumOpenAI | closed | effort medium | 54.0% | not dated | CadEval Dashboard |
| 7 | claude-3-5-sonnet-20241022Anthropic | closed | default | 48.0% | not dated | CadEval Dashboard |
| 8 | gpt-4.1-2025-04-14OpenAI | closed | default | 42.0% | not dated | CadEval Dashboard |
| 9 | gemini-1.5-pro-002Google DeepMind | closed | default | 34.0% | not dated | CadEval Dashboard |
| 10 | claude-3-5-haiku-20241022Anthropic | closed | default | 32.0% | not dated | CadEval Dashboard |
| 11 | gemini-2.0-flash-001Google DeepMind,Google | closed | default | 30.0% | not dated | CadEval Dashboard |
| 12 | gpt-4o-2024-08-06OpenAI | closed | default | 26.0% | not dated | CadEval Dashboard |
| 13 | ml-elephant | not stated | default | 20.0% | not dated | CadEval Dashboard |
| 14 | gpt-4.1-mini-2025-04-14OpenAI | closed | default | 16.0% | not dated | CadEval Dashboard |
| 15 | claude-3-haiku-20240307Anthropic | closed | default | 12.0% | not dated | CadEval Dashboard |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method