SWE-Bench verified
Re-run by Epoch AIReal issues from open-source Python repositories: the model must write a code change that makes the repository's tests pass. Verified is a subset checked by people to be solvable as stated.
- Kind
- Run independently by Epoch AI, with public logs. Our own re-runs are not built yet.
- Score shown
- Epoch's column "Best score (across scorers)", as a percentage, higher is better. Error bars as the source gives them.
- Results
- 35 in all: 6 for open-weight models, 28 for closed ones, 1 where Epoch does not say. 6 linked to a model page here.
Best open-weight result: zai-org/GLM-5.2, 78.7%, number 5 of 35 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-opus-4-7_maxAnthropic | closed | effort max | 83.5%±1.7 | 20 Apr 2026 | Epoch AI log |
| 2 | gpt-5.5-pre-release_xhighOpenAI | closed | effort xhigh | 80.6%±1.8 | 24 Apr 2026 | Epoch AI log |
| 3 | gemini-3.5-flash_highGoogle DeepMind | closed | effort high | 79.3%±1.8 | 1 Jun 2026 | Epoch AI log |
| 4 | claude-opus-4-6Anthropic | closed | default | 78.7%±1.9 | 18 Feb 2026 | Epoch AI log |
| 5 | zai-org/GLM-5.2Epoch: glm-5.2_max, Z.ai (Zhipu AI) | open | effort max | 78.7%±1.9 | 25 Jun 2026 | Epoch AI log |
| 6 | deepseek-ai/DeepSeek-V4-ProEpoch: deepseek-v4-pro_max, DeepSeek | open | effort max | 77.6%±1.9 | 18 Jun 2026 | Epoch AI log |
| 7 | qwen3.7-maxAlibaba | closed | default | 77.3%±1.9 | 18 Jun 2026 | Epoch AI log |
| 8 | gpt-5.4-2026-03-05_highOpenAI | closed | effort high | 76.9%±1.9 | 6 Mar 2026 | Epoch AI log |
| 9 | claude-opus-4-5-20251101Anthropic | closed | default | 76.7%±1.9 | 5 Feb 2026 | Epoch AI log |
| 10 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | default | 76.7%±1.9 | 8 May 2026 | Epoch AI log |
| 11 | qwen3.6-max-previewAlibaba | not stated | default | 76.7%±1.9 | 28 May 2026 | Epoch AI log |
| 12 | claude-opus-4-6Anthropic | closed | default | 75.6%±2.0 | 10 Feb 2026 | Epoch AI log |
| 13 | gemini-3.1-pro-preview-customtoolsGoogle DeepMind | closed | default | 75.6%±2.0 | 24 Feb 2026 | Epoch AI log |
| 14 | gemini-3-flash-previewGoogle DeepMind | closed | default | 75.4%±2.0 | 18 Feb 2026 | Epoch AI log |
| 15 | claude-sonnet-4-6Anthropic | closed | default | 75.2%±2.0 | 21 Feb 2026 | Epoch AI log |
| 16 | gpt-5.3-codex_highOpenAI | closed | effort high | 74.8%±2.0 | 25 Feb 2026 | Epoch AI log |
| 17 | zai-org/GLM-5.1Epoch: glm-5.1, Z.ai (Zhipu AI) | open | default | 74.2%±2.0 | 15 May 2026 | Epoch AI |
| 18 | gpt-5.2-2025-12-11_highOpenAI | closed | effort high | 73.8%±2.0 | 12 Feb 2026 | Epoch AI log |
| 19 | moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshot | open | default | 73.8%±2.0 | 17 Feb 2026 | Epoch AI log |
| 20 | gpt-5-2025-08-07_highOpenAI | closed | effort high | 73.6%±2.0 | 6 Feb 2026 | Epoch AI log |
| 21 | claude-opus-4-1-20250805Anthropic | closed | default | 73.3%±2.0 | 11 Feb 2026 | Epoch AI log |
| 22 | gemini-3-pro-previewGoogle DeepMind | closed | default | 72.9%±2.0 | 13 Feb 2026 | Epoch AI log |
| 23 | zai-org/GLM-5Epoch: glm-5, Z.ai (Zhipu AI) | open | default | 72.1%±2.1 | 15 Feb 2026 | Epoch AI log |
| 24 | gpt-5-2025-08-07_mediumOpenAI | closed | effort medium | 71.5%±2.1 | 5 Feb 2026 | Epoch AI log |
| 25 | claude-sonnet-4-5-20250929Anthropic | closed | default | 71.3%±2.1 | 5 Feb 2026 | Epoch AI log |
| 26 | claude-opus-4-20250514Anthropic | closed | default | 70.7%±2.1 | 6 Feb 2026 | Epoch AI log |
| 27 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 68.0%±2.1 | 18 Feb 2026 | Epoch AI log |
| 28 | gpt-5.1-2025-11-13_highOpenAI | closed | effort high | 65.9%±2.2 | 17 Feb 2026 | Epoch AI log |
| 29 | gpt-5-mini-2025-08-07_mediumOpenAI | closed | effort medium | 64.7%±2.2 | 1 Feb 2026 | Epoch AI log |
| 30 | o3-2025-04-16_mediumOpenAI | closed | effort medium | 62.3%±2.2 | 12 Feb 2026 | Epoch AI log |
| 31 | claude-3-7-sonnet-20250219Anthropic | closed | default | 61.0%±2.2 | 4 Feb 2026 | Epoch AI log |
| 32 | qwen3.6-plusAlibaba | closed | default | 57.9%±2.2 | 14 May 2026 | Epoch AI log |
| 33 | gemini-2.5-proGoogle DeepMind | closed | default | 57.6%±2.3 | 13 Feb 2026 | Epoch AI log |
| 34 | gpt-4.1-2025-04-14OpenAI | closed | default | 48.5%±2.3 | 8 Feb 2026 | Epoch AI log |
| 35 | gpt-4o-2024-11-20OpenAI | closed | default | 31.0%±2.1 | 11 Feb 2026 | Epoch AI log |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method