Skip to content
Weights Not Walls
Menu

GeoBench

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://geobench.org/
Score shown
Epoch's column "ACW Country %", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
Results
51 in all: 6 for open-weight models, 45 for closed ones. 5 linked to a model page here.

Best open-weight result: Qwen2.5-VL-72B-Instruct, 62.0%, number 37 of 51 results.

GeoBench results
#ModelWeightsSettingScoreDateSource
1claude-fable-5_highAnthropicclosedeffort high; Tools: None89.0%not datedgeobench.org
2gemini-3-flash-previewGoogle DeepMindclosedTools: None88.0%not datedgeobench.org
3gemini-2.5-pro-preview-05-06Google DeepMindclosedTools: None86.0%not datedgeobench.org
4gemini-3.8-flash_unknownGoogle DeepMindclosedeffort unknown; Tools: None86.0%not datedgeobench.org
5muse-spark-1.3_unknownMeta AIclosedeffort unknown; Tools: None86.0%not datedgeobench.org
6gemini-3.5-flash_unknownGoogle DeepMindclosedeffort unknown; Tools: None85.0%not datedgeobench.org
7claude-fable-5-1_highAnthropicclosedeffort high; Tools: None84.0%not datedgeobench.org
8gemini-2.5-pro-exp-03-25Google DeepMindclosedTools: Search84.0%not datedgeobench.org
9gemini-3-pro-previewGoogle DeepMindclosedTools: None84.0%not datedgeobench.org
10gemini-3.1-pro-previewGoogle DeepMindclosedTools: None84.0%not datedgeobench.org
11gemini-3.1-flash-liteGoogleclosedTools: None83.0%not datedgeobench.org
12claude-opus-4-8_highAnthropicclosedeffort high; Tools: None81.0%not datedgeobench.org
13gemini-2.5-pro-preview-05-06Google DeepMindclosedTools: Search81.0%not datedgeobench.org
14gemini-2.5-pro-preview-06-05Google DeepMindclosedTools: None81.0%not datedgeobench.org
15gpt-5-2025-08-07_mediumOpenAIclosedeffort medium; Tools: None81.0%not datedgeobench.org
16o1-2024-12-17_mediumOpenAIclosedeffort medium; Tools: None80.0%not datedGeoBench leaderboard
17gemini-2.5-pro-exp-03-25Google DeepMindclosedTools: None78.0%not datedgeobench.org
18gemini-2.0-pro-exp-02-05Google DeepMindclosedTools: None77.5%not datedgeobench.org
19gemini-2.0-flash-001Google DeepMind,GoogleclosedTools: None77.0%not datedgeobench.org
20gemini-1.5-flash-002Google DeepMindclosedTools: None76.0%not datedGeoBench leaderboard
21gemini-2.5-flash-preview-05-20Google DeepMindclosedTools: None76.0%not datedgeobench.org
22claude-opus-4-5-20251101_32KAnthropicclosedTools: None75.0%not datedgeobench.org
23claude-sonnet-5_highAnthropicclosedeffort high; Tools: None74.0%not datedgeobench.org
24o3-2025-04-16_mediumOpenAIclosedeffort medium; Tools: None74.0%not datedGeoBench leaderboard
25claude-sonnet-4-6_32KAnthropicclosedTools: None73.0%not datedgeobench.org
26gemini-2.5-flash-preview-04-17Google DeepMindclosedTools: None73.0%not datedGeoBench leaderboard
27gpt-5.6-sol_highOpenAIclosedeffort high; Tools: None73.0%not datedgeobench.org
28gpt-4.1-2025-04-14OpenAIclosedTools: None72.0%not datedGeoBench leaderboard
29gpt-4o-2024-11-20OpenAIclosedTools: None71.0%not datedGeoBench leaderboard
30gemini-2.0-flash-thinking-exp-01-21Google DeepMind,GoogleclosedTools: None70.0%not datedgeobench.org
31claude-3-7-sonnet-20250219AnthropicclosedTools: None68.0%not datedGeoBench leaderboard
32claude-3-7-sonnet-20250219_15KAnthropicclosedTools: None65.0%not datedgeobench.org
33gpt-4o-mini-2024-07-18OpenAIclosedTools: None64.0%not datedGeoBench leaderboard
34o4-mini-2025-04-16_highOpenAIclosedeffort high; Tools: None64.0%not datedGeoBench leaderboard
35o4-mini-2025-04-16_mediumOpenAIclosedeffort medium; Tools: None64.0%not datedGeoBench leaderboard
36claude-3-5-sonnet-20241022AnthropicclosedTools: None62.0%not datedGeoBench leaderboard
37Qwen2.5-VL-72B-InstructAlibabano page matchopenTools: None62.0%not datedGeoBench leaderboard
38grok-4.7_unknownxAIclosedeffort unknown; Tools: None61.0%not datedgeobench.org
39claude-sonnet-4-5-20250929_32KAnthropicclosedTools: None60.0%not datedgeobench.org
40o3-2025-04-16_highOpenAIclosedeffort high; Tools: None60.0%not datedGeoBench leaderboard
41grok-4.3_unknownxAIclosedeffort unknown; Tools: None56.0%not datedgeobench.org
42google/gemma-3-27b-itEpoch: gemma-3-27b-it, Google DeepMindopenTools: None52.0%not datedGeoBench leaderboard
43meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AIopenTools: None52.0%not datedGeoBench leaderboard
44meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: Llama-4-Maverick-17B-128E-Instruct, Meta AIopenTools: None52.0%not datedGeoBench leaderboard
45claude-opus-4-20250514_32KAnthropicclosedTools: None49.0%not datedGeoBench leaderboard
46grok-4-0709xAIclosedTools: None45.0%not datedgeobench.org
47Qwen/Qwen3.8-27BEpoch: qwen3.8-27b, AlibabaopenTools: None38.0%not datedgeobench.org
48claude-sonnet-4-20250514AnthropicclosedTools: None37.0%not datedGeoBench leaderboard
49mistralai/Pixtral-12B-2409Epoch: Pixtral-12B-2409, Mistral AIopenTools: None37.0%not datedGeoBench leaderboard
50claude-3-5-haiku-20241022AnthropicclosedTools: None34.0%not datedGeoBench leaderboard
51claude-sonnet-4-20250514_32KAnthropicclosedTools: None32.0%not datedGeoBench leaderboard

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method