Skip to content
Weights Not Walls
Menu

OSWorld

Reported

Tasks done in a real desktop operating system through the screen, keyboard and mouse.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://os-world.github.io/
Score shown
Epoch's column "Score", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
Results
20 in all: 3 for open-weight models, 14 for closed ones, 3 where Epoch does not say. 1 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K2.5, 63.3%, number 3 of 20 results.

OSWorld results
#ModelWeightsSettingScoreDateSource
1claude-sonnet-4-6AnthropicclosedAgent: claude-sonnet-4-6 (100 steps)72.1%not datedOS World Website
2claude-opus-4-5-20251101AnthropicclosedAgent: Claude Opus 4.566.3%25 Nov 2025Anthropic annoucement
3moonshotai/Kimi-K2.5Epoch: kimi-k2.5, MoonshotopenAgent: Kimi-K2.563.3%3 Feb 2026OS World Website
4claude-sonnet-4-5-20250929AnthropicclosedAgent: claude-sonnet-4-5-20250929 (100 steps)62.9%30 Oct 2025OS World Website
5claude-sonnet-4-5-20250929AnthropicclosedAgent: claude-sonnet-4-5-20250929 (50 steps)58.1%30 Oct 2025OS World Website
6claude-sonnet-4-20250514AnthropicclosedAgent: claude-4-sonnet-20250514 (50 steps)43.9%27 Jul 2025OS World Website
7claude-sonnet-4-5-20250929AnthropicclosedAgent: claude-sonnet-4-5-20250929 (15 steps)42.9%30 Oct 2025OS World Website
8claude-sonnet-4-20250514AnthropicclosedAgent: claude-4-sonnet-20250514 (100 steps)41.4%27 Jul 2025OS World Website
9claude-3-7-sonnet-20250219AnthropicclosedAgent: claude-3-7-sonnet-20250219 (50 steps)35.8%27 Jul 2025OS World Website
10claude-3-7-sonnet-20250219AnthropicclosedAgent: claude-3-7-sonnet-20250219 (100 steps)35.6%27 Jul 2025OS World Website
11computer-use-preview-2025-03-11not statedAgent: computer-use-preview (50 steps)31.3%27 Jul 2025OS World Website
12claude-sonnet-4-20250514AnthropicclosedAgent: claude-4-sonnet-20250514 (15 steps)31.2%27 Jul 2025OS World Website
13computer-use-preview-2025-03-11not statedAgent: computer-use-preview (100 steps)30.5%27 Jul 2025OS World Website
14claude-3-7-sonnet-20250219AnthropicclosedAgent: claude-3-7-sonnet-20250219 (15 steps)27.1%27 Jul 2025OS World Website
15computer-use-preview-2025-03-11not statedAgent: computer-use-preview (15 steps)26.0%27 Jul 2025OS World Website
16o3-2025-04-16_mediumOpenAIclosedeffort medium; Agent: o3 (100 steps)23.0%27 Jul 2025OS World Website
17o3-2025-04-16_mediumOpenAIclosedeffort medium; Agent: o3 (50 steps)17.2%27 Jul 2025OS World Website
18o3-2025-04-16_mediumOpenAIclosedeffort medium; Agent: o3 (15 steps)9.1%27 Jul 2025OS World Website
19Qwen2.5-VL-72B-InstructAlibabano page matchopenAgent: qwen2.5-vl-72b-instruct (100 steps)5.0%27 Jul 2025OS World Website
20Qwen2.5-VL-72B-InstructAlibabano page matchopenAgent: qwen2.5-vl-72b-instruct (15 steps)4.4%27 Jul 2025OS World Website

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method