Skip to content
Weights Not Walls
Menu

Fiction.LiveBench

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://fiction.live/stories/Fiction-liveBench-Mar-14-2025/oQdzQvKHw8JyXbN87/home
Score shown
Epoch's column "16k token score", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
Results
61 in all: 26 for open-weight models, 33 for closed ones, 2 where Epoch does not say. 25 linked to a model page here.

Best open-weight result: moonshotai/Kimi-K2.5, 86.1%, number 7 of 61 results.

Fiction.LiveBench results
#ModelWeightsSettingScoreDateSource
1gpt-5-2025-08-07_mediumOpenAIclosedeffort medium97.2%not datedFiction.live leaderboard
2o3-pro-2025-06-10_mediumOpenAIclosedeffort medium97.2%not datedFiction.live leaderboard
3grok-4-0709xAIcloseddefault94.4%not datedFiction.live leaderboard
4grok-4-fastxAIcloseddefault94.4%not datedFiction.live leaderboard
5gemini-2.5-pro-preview-06-05Google DeepMindcloseddefault91.7%not datedFiction.live leaderboard
6o3-2025-04-16_mediumOpenAIclosedeffort medium88.9%not datedFiction.live leaderboard
7moonshotai/Kimi-K2.5Epoch: kimi-k2.5, Moonshotopendefault86.1%not datedFiction.live leaderboard
8Qwen/QwQ-32BEpoch: chutes/QwQ-32B, Alibabaopendefault83.3%not datedFiction.live leaderboard
9claude-3-7-sonnet-20250219_8KAnthropiccloseddefault83.3%not datedFiction.live leaderboard
10deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp_high, DeepSeekopeneffort high83.3%not datedFiction.live leaderboard
11o1-2024-12-17_mediumOpenAIclosedeffort medium83.3%not datedFiction.live leaderboard
12gemini-2.5-flash-preview-05-20Google DeepMindcloseddefault77.8%not datedFiction.live leaderboard
13o4-mini-2025-04-16_mediumOpenAIclosedeffort medium77.8%not datedFiction.live leaderboard
14deepseek-ai/DeepSeek-R1-0528Epoch: chutes/DeepSeek-R1-0528, DeepSeekopendefault75.0%not datedFiction.live leaderboard
15Qwen/Qwen3-235B-A22B-Thinking-2507Epoch: chutes/Qwen3-235B-A22B-Thinking-2507, Alibabaopendefault75.0%not datedFiction.live leaderboard
16Qwen/Qwen3-32BEpoch: chutes/Qwen3-32B, Alibabaopendefault74.2%not datedFiction.live leaderboard
17deepseek-ai/DeepSeek-R1Epoch: DeepSeek-R1, DeepSeekopendefault69.4%not datedFiction.live leaderboard
18gpt-5-mini-2025-08-07_mediumOpenAIclosedeffort medium69.4%not datedFiction.live leaderboard
19MiniMaxAI/MiniMax-M1-80kEpoch: MiniMax-M1-80k, MiniMaxopendefault69.4%not datedFiction.live leaderboard
20Qwen/Qwen3-235B-A22BEpoch: chutes/Qwen3-235B-A22B, Alibabaopendefault67.7%not datedFiction.live leaderboard
21chatgpt-4o-01-29-2025OpenAIcloseddefault66.7%not datedFiction.live leaderboard
22moonshotai/Kimi-K2-Instruct-0905Epoch: fireworks/Kimi-K2-Instruct-0905, Moonshotopendefault66.7%not datedFiction.live leaderboard
23gemini-2.5-pro-exp-03-25Google DeepMindcloseddefault66.7%not datedFiction.live leaderboard
24gemini-2.5-pro-preview-03-25Google DeepMindcloseddefault66.7%not datedFiction.live leaderboard
25gemini-2.5-pro-preview-05-06Google DeepMindcloseddefault66.7%not datedFiction.live leaderboard
26grok-3-mini-beta_mediumxAIclosedeffort medium66.7%not datedFiction.live leaderboard
27qwen-max-2025-01-25Alibabacloseddefault66.7%not datedFiction.live leaderboard
28qwen3-max-2025-09-23Alibabacloseddefault66.7%not datedFiction.live leaderboard
29gpt-4.1-2025-04-14OpenAIcloseddefault63.9%not datedFiction.live leaderboard
30gpt-4.5-preview-2025-02-27OpenAIcloseddefault63.9%not datedFiction.live leaderboard
31Qwen/Qwen3-14BEpoch: chutes/Qwen3-14B, Alibabaopendefault62.5%not datedFiction.live leaderboard
32Qwen/Qwen3-8BEpoch: chutes/Qwen3-8B, Alibabaopendefault62.1%not datedFiction.live leaderboard
33claude-opus-4-20250514Anthropiccloseddefault61.1%not datedFiction.live leaderboard
34gemini-2.0-flash-001Google DeepMind,Googlecloseddefault61.1%not datedFiction.live leaderboard
35moonshotai/Kimi-K2-InstructEpoch: Kimi-K2-Instruct, Moonshotopendefault61.1%not datedFiction.live leaderboard
36chutes/GLM-4.5-FP8not stateddefault58.3%not datedFiction.live leaderboard
37grok-3-betaxAIcloseddefault58.3%not datedFiction.live leaderboard
38deepinfra/Qwen3-Next-80B-A3B-Instructnot stateddefault55.6%not datedFiction.live leaderboard
39Qwen/Qwen3-235B-A22B-Instruct-2507Epoch: parasail-qwen3-235b-a22b-instruct-2507, Alibabaopendefault52.9%not datedFiction.live leaderboard
40deepseek-ai/DeepSeek-V3.1Epoch: DeepSeek-V3.1, DeepSeekopendefault52.8%not datedFiction.live leaderboard
41deepseek-ai/DeepSeek-V3.2-ExpEpoch: DeepSeek-V3.2-Exp, DeepSeekopendefault52.8%not datedFiction.live leaderboard
42gemini-2.0-flash-thinking-exp-01-21Google DeepMind,Googlecloseddefault52.8%not datedFiction.live leaderboard
43deepseek-ai/DeepSeek-V3-0324Epoch: chutes/DeepSeek-V3-0324, DeepSeekopendefault50.0%not datedFiction.live leaderboard
44claude-3-7-sonnet-20250219Anthropiccloseddefault50.0%not datedFiction.live leaderboard
45o3-mini-2025-01-31_mediumOpenAIclosedeffort medium50.0%not datedFiction.live leaderboard
46gemini-2.5-flash-lite-preview-06-17-thinkingGoogle DeepMindcloseddefault47.2%not datedFiction.live leaderboard
47gemini-2.5-flash-preview-04-17Google DeepMindcloseddefault47.2%not datedFiction.live leaderboard
48claude-sonnet-4-20250514Anthropiccloseddefault46.9%not datedFiction.live leaderboard
49meta-llama/Llama-4-Maverick-17B-128E-InstructEpoch: chutes/Llama-4-Maverick-17B-128E-Instruct, Meta AIopendefault46.2%not datedFiction.live leaderboard
50openai/gpt-oss-120bEpoch: chutes/gpt-oss-120b_high, OpenAIopeneffort high44.4%not datedFiction.live leaderboard
51gpt-4.1-mini-2025-04-14OpenAIcloseddefault44.4%not datedFiction.live leaderboard
52gpt-5-nano-2025-08-07_mediumOpenAIclosedeffort medium44.4%not datedFiction.live leaderboard
53Qwen/Qwen3-Next-80B-A3B-InstructEpoch: chutes/Qwen3-Next-80B-A3B-Instructopendefault41.7%not datedFiction.live leaderboard
54gemini-2.0-pro-exp-02-05Google DeepMindcloseddefault41.7%not datedFiction.live leaderboard
55Qwen/Qwen3-30B-A3BEpoch: chutes/Qwen3-30B-A3B, Alibabaopendefault40.6%not datedFiction.live leaderboard
56openai/gpt-oss-120bEpoch: chutes/gpt-oss-120b, OpenAIopendefault36.1%not datedFiction.live leaderboard
57chutes/Llama-4-Scout-17B-16E InstructMeta AIno page matchopendefault36.0%not datedFiction.live leaderboard
58google/gemma-3-27b-itEpoch: chutes/Gemma-3-27b-It, Google DeepMindopendefault33.3%not datedFiction.live leaderboard
59meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopendefault33.3%not datedFiction.live leaderboard
60gpt-4.1-nano-2025-04-14OpenAIcloseddefault25.0%not datedFiction.live leaderboard
61nvidia/NVIDIA-Nemotron-Nano-9B-v2Epoch: nvidia-nemotron-nano-9b-v2opendefault25.0%not datedFiction.live leaderboard

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method