OSWorld 2.0
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://osworld-v2.xlang.ai/
- Score shown
- Epoch's column "Binary accuracy", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
- Results
- 16 in all: 2 for open-weight models, 14 for closed ones. 2 linked to a model page here.
Best open-weight result: moonshotai/Kimi-K2.6, 4.6%, number 14 of 16 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | claude-opus-5_maxAnthropic | closed | effort max; Reasoning: max; Tool setting: batch tool; Step budget: 500 | 31.4% | not dated | osworld-v2.xlang.ai |
| 2 | claude-opus-5_xhighAnthropic | closed | effort xhigh; Reasoning: xhigh; Tool setting: batch tool; Step budget: 500 | 30.2% | not dated | osworld-v2.xlang.ai |
| 3 | claude-opus-5_highAnthropic | closed | effort high; Reasoning: high; Tool setting: batch tool; Step budget: 500 | 29.0% | not dated | osworld-v2.xlang.ai |
| 4 | gpt-5.6-sol_maxOpenAI | closed | effort max; Reasoning: max; Tool setting: batch tool; Step budget: 500 | 27.3% | not dated | osworld-v2.xlang.ai |
| 5 | claude-opus-5_mediumAnthropic | closed | effort medium; Reasoning: medium; Tool setting: batch tool; Step budget: 500 | 25.3% | not dated | osworld-v2.xlang.ai |
| 6 | claude-opus-5_lowAnthropic | closed | effort low; Reasoning: low; Tool setting: batch tool; Step budget: 500 | 22.3% | not dated | osworld-v2.xlang.ai |
| 7 | claude-opus-4-8_maxAnthropic | closed | effort max; Reasoning: max; Tool setting: batched tool; Step budget: 500 | 20.6% | not dated | osworld-v2.xlang.ai |
| 8 | claude-opus-4-8_maxAnthropic | closed | effort max; Reasoning: max; Tool setting: standard; Step budget: 500 | 18.5% | not dated | osworld-v2.xlang.ai |
| 9 | claude-opus-4-7_maxAnthropic | closed | effort max; Reasoning: max; Tool setting: batched tool; Step budget: 500 | 18.2% | not dated | osworld-v2.xlang.ai |
| 10 | claude-opus-4-7_maxAnthropic | closed | effort max; Reasoning: max; Tool setting: standard; Step budget: 500 | 13.9% | not dated | osworld-v2.xlang.ai |
| 11 | gpt-5.5_xhighOpenAI | closed | effort xhigh; Reasoning: xhigh; Tool setting: batch tool; Step budget: 500 | 13.0% | not dated | osworld-v2.xlang.ai |
| 12 | claude-sonnet-4-6_mediumAnthropic | closed | effort medium; Reasoning: medium; Tool setting: standard; Step budget: 500 | 9.3% | not dated | osworld-v2.xlang.ai |
| 13 | claude-sonnet-4-6_maxAnthropic | closed | effort max; Reasoning: max; Tool setting: standard; Step budget: 500 | 8.3% | not dated | osworld-v2.xlang.ai |
| 14 | moonshotai/Kimi-K2.6Epoch: kimi-k2.6, Moonshot | open | Reasoning: enabled; Tool setting: standard; Step budget: 500 | 4.6% | not dated | osworld-v2.xlang.ai |
| 15 | MiniMaxAI/MiniMax-M3Epoch: MiniMax-M3, MiniMax | open | Reasoning: enabled; Tool setting: standard; Step budget: 500 | 4.6% | not dated | osworld-v2.xlang.ai |
| 16 | qwen3.7-plusAlibaba | closed | Reasoning: thinking; Tool setting: standard; Step budget: 500 | 2.8% | not dated | osworld-v2.xlang.ai |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method