Skip to content
Weights Not Walls
Menu

SWE-Bench verified

Re-run by Epoch AI

Real issues from open-source Python repositories: the model must write a code change that makes the repository's tests pass. Verified is a subset checked by people to be solvable as stated.

Kind
Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
Score shown
Epoch's column "Best score (across scorers)", as a percentage, higher is better. Error bars as the source gives them.
Results
35 in all: 6 for open-weight models, 28 for closed ones, 1 where Epoch does not say. 6 linked to a model page here.

Best open-weight result: zai-org/GLM-5.2, 78.7%, number 5 of 35 results.

SWE-Bench verified results
#ModelWeightsSettingScoreDateSource
1claude-opus-4-7_maxAnthropicclosedeffort max83.5%±1.720 Apr 2026Epoch AI log
2gpt-5.5-pre-release_xhighOpenAIclosedeffort xhigh80.6%±1.824 Apr 2026Epoch AI log
3gemini-3.5-flash_highGoogle DeepMindclosedeffort high79.3%±1.81 Jun 2026Epoch AI log
4claude-opus-4-6Anthropiccloseddefault78.7%±1.918 Feb 2026Epoch AI log
5zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI)openeffort max78.7%±1.925 Jun 2026Epoch AI log
6deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeekopeneffort max77.6%±1.918 Jun 2026Epoch AI log
7qwen3.7-maxAlibabacloseddefault77.3%±1.918 Jun 2026Epoch AI log
8gpt-5.4-2026-03-05_highOpenAIclosedeffort high76.9%±1.96 Mar 2026Epoch AI log
9claude-opus-4-5-20251101Anthropiccloseddefault76.7%±1.95 Feb 2026Epoch AI log
10moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshotopendefault76.7%±1.98 May 2026Epoch AI log
11qwen3.6-max-previewAlibabanot stateddefault76.7%±1.928 May 2026Epoch AI log
12claude-opus-4-6Anthropiccloseddefault75.6%±2.010 Feb 2026Epoch AI log
13gemini-3.1-pro-preview-customtoolsGoogle DeepMindcloseddefault75.6%±2.024 Feb 2026Epoch AI log
14gemini-3-flash-previewGoogle DeepMindcloseddefault75.4%±2.018 Feb 2026Epoch AI log
15claude-sonnet-4-6Anthropiccloseddefault75.2%±2.021 Feb 2026Epoch AI log
16gpt-5.3-codex_highOpenAIclosedeffort high74.8%±2.025 Feb 2026Epoch AI log
17zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI)opendefault74.2%±2.015 May 2026Epoch AI
18gpt-5.2-2025-12-11_highOpenAIclosedeffort high73.8%±2.012 Feb 2026Epoch AI log
19moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshotopendefault73.8%±2.017 Feb 2026Epoch AI log
20gpt-5-2025-08-07_highOpenAIclosedeffort high73.6%±2.06 Feb 2026Epoch AI log
21claude-opus-4-1-20250805Anthropiccloseddefault73.3%±2.011 Feb 2026Epoch AI log
22gemini-3-pro-previewGoogle DeepMindcloseddefault72.9%±2.013 Feb 2026Epoch AI log
23zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI)opendefault72.1%±2.115 Feb 2026Epoch AI log
24gpt-5-2025-08-07_mediumOpenAIclosedeffort medium71.5%±2.15 Feb 2026Epoch AI log
25claude-sonnet-4-5-20250929Anthropiccloseddefault71.3%±2.15 Feb 2026Epoch AI log
26claude-opus-4-20250514Anthropiccloseddefault70.7%±2.16 Feb 2026Epoch AI log
27gpt-5.1-2025-11-13_highOpenAIclosedeffort high68.0%±2.118 Feb 2026Epoch AI log
28gpt-5.1-2025-11-13_highOpenAIclosedeffort high65.9%±2.217 Feb 2026Epoch AI log
29gpt-5-mini-2025-08-07_mediumOpenAIclosedeffort medium64.7%±2.21 Feb 2026Epoch AI log
30o3-2025-04-16_mediumOpenAIclosedeffort medium62.3%±2.212 Feb 2026Epoch AI log
31claude-3-7-sonnet-20250219Anthropiccloseddefault61.0%±2.24 Feb 2026Epoch AI log
32qwen3.6-plusAlibabacloseddefault57.9%±2.214 May 2026Epoch AI log
33gemini-2.5-proGoogle DeepMindcloseddefault57.6%±2.313 Feb 2026Epoch AI log
34gpt-4.1-2025-04-14OpenAIcloseddefault48.5%±2.38 Feb 2026Epoch AI log
35gpt-4o-2024-11-20OpenAIcloseddefault31.0%±2.111 Feb 2026Epoch AI log

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method