Cybench
ReportedCapture-the-flag security tasks taken from professional competitions.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://cybench.github.io/
- Score shown
- Epoch's column "Unguided % Solved", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
- Results
- 22 in all: 3 for open-weight models, 19 for closed ones. 3 linked to a model page here.
Best open-weight result: meta-llama/Llama-3.1-405B-Instruct, 7.5%, number 20 of 22 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-opus-4-6_unknownAnthropic | closed | effort unknown | 93.0% | not dated | Opus 4.6 System Card |
| 2 | claude-opus-4-5-20251101_unknownAnthropic | closed | effort unknown | 82.0% | not dated | Opus 4.5 System Card |
| 3 | claude-sonnet-4-5-20250929_unknownAnthropic | closed | effort unknown | 60.0% | not dated | Opus 4.5 System Card |
| 4 | claude-sonnet-4-5-20250929Anthropic | closed | default | 55.0% | not dated | Cybench leaderboard |
| 5 | grok-4-0709xAI | closed | default | 43.0% | not dated | Grok 4 Model Card |
| 6 | claude-opus-4-1-20250805Anthropic | closed | default | 42.0% | not dated | Cybench leaderboard |
| 7 | grok-4-1xAI | closed | default | 39.0% | not dated | Grok 4.1 Model Card |
| 8 | claude-opus-4-20250514Anthropic | closed | default | 38.0% | not dated | Cybench leaderboard |
| 9 | claude-sonnet-4-20250514Anthropic | closed | default | 35.0% | not dated | Cybench leaderboard |
| 10 | grok-4-fastxAI | closed | default | 30.0% | not dated | Grok 4 Fast Model Card |
| 11 | o3-mini-2025-01-31_mediumOpenAI | closed | effort medium | 22.5% | not dated | Cybench leaderboard |
| 12 | claude-3-7-sonnet-20250219Anthropic | closed | default | 20.0% | not dated | Cybench leaderboard |
| 13 | claude-3-5-sonnet-20240620Anthropic | closed | default | 17.5% | not dated | Cybench leaderboard |
| 14 | gpt-4.5-preview-2025-02-27OpenAI | closed | default | 17.5% | not dated | Cybench leaderboard |
| 15 | gpt-4o-2024-11-20OpenAI | closed | default | 12.5% | not dated | Cybench leaderboard |
| 16 | claude-3-opus-20240229Anthropic | closed | default | 10.0% | not dated | Cybench leaderboard |
| 17 | o1-mini-2024-09-12_mediumOpenAI | closed | effort medium | 10.0% | not dated | Cybench leaderboard |
| 18 | o1-preview-2024-09-12OpenAI | closed | default | 10.0% | not dated | Cybench leaderboard |
| 19 | gemini-1.5-pro-001-feb24Google DeepMind | closed | default | 7.5% | not dated | Cybench leaderboard |
| 20 | meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AI | open | default | 7.5% | not dated | Cybench leaderboard |
| 21 | mistralai/Mixtral-8x22B-Instruct-v0.1Epoch: Mixtral-8x22B-Instruct-v0.1, Mistral AI | open | default | 7.5% | not dated | Cybench leaderboard |
| 22 | meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AI | open | default | 5.0% | not dated | Cybench leaderboard |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method