Skip to content
Weights Not Walls
Menu

MMLU

Reported

Multiple-choice questions across 57 subjects, from school level to professional level.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Score shown
Epoch's column "EM", as a percentage, higher is better; random guessing scores about 25.0%. The source gives no error bars.
Results
217 in all: 159 for open-weight models, 51 for closed ones, 7 where Epoch does not say. 79 linked to a model page here.

Best open-weight result: deepseek-ai/DeepSeek-V3, 87.2%, number 3 of 217 results.

MMLU results
#ModelWeightsSettingScoreDateSource
1gpt-4o-2024-11-20OpenAIclosedShots: 588.1%not datedPhi-4 Technical Report
2claude-3-5-sonnet-20241022AnthropicclosedShots: 587.3%not datedStanford CRFM Leaderboard
3deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopenShots: 587.2%not datedStanford CRFM Leaderboard
4deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopenShots: 587.1%not datedDeepSeek-V3 Technical Report
5gemini-1.5-pro-002Google DeepMindclosedShots: 586.9%not datedStanford CRFM Leaderboard
6claude-3-5-sonnet-20240620AnthropicclosedShots: 586.5%not datedStanford CRFM Leaderboard
7gpt-4-0314OpenAIclosedShots: 586.4%not datedGPT-4 Technical Report
8meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopenShots: 586.3%not datedPhi-4 Technical Report
9gemini-1.5-pro-001Google DeepMindclosedShots: 585.9%not datedGemini 1.5 Report
10Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, AlibabaopenShots: 585.3%not datedPhi-4 Technical Report
11Qwen2.5-72BAlibabano page matchopenShots: 585.0%not datedDeepSeek-V3 Technical Report
12microsoft/phi-4Epoch: phi-4, Microsoft ResearchopenShots: 584.8%not datedPhi-4 Technical Report
13claude-3-opus-20240229AnthropicclosedShots: 584.6%not datedStanford CRFM Leaderboard
14meta-llama/Llama-3.1-405B-InstructEpoch: Llama-3.1-405B-Instruct, Meta AIopenShots: 584.5%not datedStanford CRFM Leaderboard
15Llama-3.1-405BMeta AIno page matchopenShots: 584.4%not datedDeepSeek-V3 Technical Report
16gpt-4o-2024-08-06OpenAIclosedShots: 584.3%not datedStanford CRFM Leaderboard
17gpt-4o-2024-05-13OpenAIclosedShots: 584.2%not datedStanford CRFM Leaderboard
18Qwen/Qwen2.5-72B-InstructEpoch: qwen2.5-72b-instruct, AlibabaopenShots: 583.4%not datedStanford CRFM Leaderboard
19gemini-1.5-pro-001-feb24Google DeepMindclosedShots: 582.7%not datedStanford CRFM Leaderboard
20gpt-4-0613OpenAIclosedShots: 582.4%not datedStanford CRFM Leaderboard
21Qwen/Qwen2-72B-InstructEpoch: qwen2-72b-instruct, AlibabaopenShots: 582.4%not datedStanford CRFM Leaderboard
22amazon.nova-pro-v1:0AmazonclosedShots: 582.0%not datedStanford CRFM Leaderboard
23gemini-1.5-pro-001-feb24Google DeepMindclosedShots: 581.9%not datedGemini 1.5 Report
24gpt-4o-mini-2024-07-18OpenAIclosedShots: 581.8%not datedPhi-4 Technical Report
25gpt-4-turbo-2024-04-09OpenAIclosedShots: 581.3%not datedStanford CRFM Leaderboard
26meta-llama/Llama-3.2-90B-Vision-InstructEpoch: Llama-3.2-90B-Vision-Instruct, Meta AIopenShots: 580.3%not datedStanford CRFM Leaderboard
27meta-llama/Llama-3.1-70B-InstructEpoch: Llama-3.1-70B-Instruct, Meta AIopenShots: 580.1%not datedStanford CRFM Leaderboard
28mistral-large-2407Mistral AIno page matchopenShots: 580.0%not datedStanford CRFM Leaderboard
29Qwen/Qwen2.5-14B-InstructEpoch: qwen2.5-14b-instruct, AlibabaopenShots: 579.9%not datedPhi-4 Technical Report
30gemini-2.0-flash-expGoogle DeepMind,GoogleclosedShots: 579.7%not datedStanford CRFM Leaderboard
31gpt-4-turboOpenAIclosedShots: 579.6%not datedStanford CRFM Leaderboard
32meta-llama/Meta-Llama-3-70B-InstructEpoch: Meta-Llama-3-70B-Instruct, Meta AIopenShots: 579.3%not datedStanford CRFM Leaderboard
33Yi-large01.AIclosedShots: 579.3%not datedStanford CRFM Leaderboard
34meta-llama/Llama-3.3-70B-InstructEpoch: Llama-3.3-70B-Instruct, Meta AIopenShots: 579.1%not datedStanford CRFM Leaderboard
35Qwen2.5-Coder-32BAlibabano page matchopendefault79.1%not datedQwen2.5-Coder Technical Report
36claude-2.0AnthropicclosedShots: 578.5%not datedModel Card and Evaluations for Claude Models
37DeepSeek-V2DeepSeekno page matchopenShots: 578.4%not datedDeepSeek-V3 Technical Report
38microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, MicrosoftopenShots: 578.0%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
39gemini-1.5-flash-001Google DeepMindclosedShots: 577.9%not datedStanford CRFM Leaderboard
40microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, MicrosoftopenShots: 577.9%not datedPhi-4 Technical Report
41gemini-1.5-flash-0514Google DeepMindclosedShots: 577.8%not datedStanford CRFM Leaderboard
42mistralai/Mixtral-8x22B-v0.1Epoch: Mixtral-8x22B-v0.1, Mistral AIopenShots: 577.8%not datedStanford CRFM Leaderboard
43microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, MicrosoftopenShots: 577.5%not datedStanford CRFM Leaderboard
44amazon.nova-lite-v1:0AmazonclosedShots: 577.0%not datedStanford CRFM Leaderboard
45claude-1.3AnthropicclosedShots: 577.0%not datedModel Card and Evaluations for Claude Models
46gpt-4o-mini-2024-07-18OpenAIclosedShots: 576.7%not datedStanford CRFM Leaderboard
47Yi-34B01.AIno page matchopenShots: 576.3%not datedYi: Open Foundation Models by 01.AI
48Yi-34B01.AIno page matchopenShots: 576.2%not datedStanford CRFM Leaderboard
49claude-3-sonnet-20240229AnthropicclosedShots: 575.9%not datedStanford CRFM Leaderboard
50google/gemma-2-27b-itEpoch: gemma-2-27b-it, Google DeepMindopenShots: 575.7%not datedStanford CRFM Leaderboard
51microsoft/Phi-3-small-8k-instructEpoch: Phi-3-small-8k-instruct, MicrosoftopenShots: 575.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
52Qwen/Qwen2.5-Coder-14BEpoch: Qwen2.5-Coder-14Bopendefault75.2%not datedQwen2.5-Coder Technical Report
53qwen1.5-32BAlibabano page matchopenShots: 574.4%not datedStanford CRFM Leaderboard
54claude-3-5-haiku-20241022AnthropicclosedShots: 574.3%not datedStanford CRFM Leaderboard
55gemini-1.5-flash-002Google DeepMindclosedShots: 573.9%not datedStanford CRFM Leaderboard
56claude-3-haiku-20240307AnthropicclosedShots: 573.8%not datedStanford CRFM Leaderboard
57claude-2.1AnthropicclosedShots: 573.5%not datedStanford CRFM Leaderboard
5801-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AIopenShots: 573.5%not datedYi: Open Foundation Models by 01.AI
59claude-instant-1.1AnthropicclosedShots: 573.4%not datedModel Card and Evaluations for Claude Models
60claude-instant-1.1AnthropicclosedShots: 573.4%not datedReleasing Claude Instant 1.2
61claude-instant-1.2AnthropicclosedShots: 573.2%not datedReleasing Claude Instant 1.2
62Qwen/Qwen2.5-7B-InstructEpoch: qwen2.5-7b-instruct, AlibabaopenShots: 572.9%not datedStanford CRFM Leaderboard
63Inflection-1Inflection AIclosedShots: 572.7%not datedInflection-1
64google/gemma-2-9b-itEpoch: gemma-2-9b-it, Google DeepMindopenShots: 572.1%not datedStanford CRFM Leaderboard
65gpt-3.5-turbo-1106OpenAIclosedShots: 571.4%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
66amazon.nova-micro-v1:0AmazonclosedShots: 570.8%not datedStanford CRFM Leaderboard
67falcon-180BTechnology Innovation Instituteno page matchopenShots: 570.6%not datedThe Falcon Series of Open Language Models
68mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AIopenShots: 570.6%not datedMixtral of Experts
69mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AIopenShots: 570.5%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
70gemini-1.0-pro-001Google DeepMindclosedShots: 570.0%not datedStanford CRFM Leaderboard
71text-davinci-002OpenAIclosedShots: 570.0%not datedGPT-4 Technical Report
72meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 569.9%not datedMixtral of Experts
73meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 569.8%not datedQwen Technical Report
74meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 569.5%not datedStanford CRFM Leaderboard
75CohereLabs/c4ai-command-r-plus-08-2024Epoch: c4ai-command-r-plus-08-2024, Cohere,Cohere Labs (formerly Cohere for AI)openShots: 569.4%not datedStanford CRFM Leaderboard
76PaLM 540BGoogle ResearchclosedShots: 569.3%not datedLLaMA: Open and Efficient Foundation Language Models
77gpt-3.5-turbo-0613OpenAIclosedShots: 568.9%not datedStanford CRFM Leaderboard
78meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 568.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
79claude-instant-1.2AnthropicclosedShots: 568.8%not datedStanford CRFM Leaderboard
80meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AIopenShots: 568.8%not datedStanford CRFM Leaderboard
81mistral-large-2402Mistral AIclosedShots: 568.8%not datedStanford CRFM Leaderboard
82microsoft/Phi-3-mini-4k-instructEpoch: Phi-3-mini-4k-instruct, MicrosoftopenShots: 568.8%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
83mistral-small-2402not statedShots: 568.7%not datedStanford CRFM Leaderboard
84qwen1.5-14BAlibabano page matchopenShots: 568.6%not datedStanford CRFM Leaderboard
85stabilityai/StableBeluga2Epoch: StableBeluga2, Stability AIopenShots: 568.6%not datedQwen Technical Report
8601-ai/Yi-9BEpoch: Yi-9BopenShots: 568.4%not datedYi: Open Foundation Models by 01.AI
87Qwen2.5-Coder-7BAlibabano page matchopendefault68.0%not datedQwen2.5-Coder Technical Report
8801-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AIopenShots: 067.6%not datedYi: Open Foundation Models by 01.AI
89Chinchilla (70B)DeepMindclosedShots: 567.5%not datedLLaMA: Open and Efficient Foundation Language Models
90gpt-3.5-turbo-0125OpenAIclosedShots: 567.3%not datedStanford CRFM Leaderboard
91meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AIopenShots: 566.5%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
92Qwen-14BAlibabano page matchopenShots: 566.3%not datedNemotron-4 15B Technical Report
93gemma-7bGoogle DeepMindno page matchopenShots: 566.1%not datedStanford CRFM Leaderboard
94CohereLabs/c4ai-command-r-08-2024Epoch: c4ai-command-r-08-2024openShots: 565.2%not datedStanford CRFM Leaderboard
95Qwen/Qwen-14B-ChatEpoch: Qwen-14B-Chat, AlibabaopenShots: 565.0%not datedYi: Open Foundation Models by 01.AI
96gemma-7bGoogle DeepMindno page matchopenShots: 564.3%not datedGemma: Open Models Based on Gemini Research and Technology
97gemma-7bGoogle DeepMindno page matchopenShots: 564.3%not datedNemotron-4 15B Technical Report
98bigcode/starcoder2-15bEpoch: starcoder2-15b, Hugging Face,ServiceNow,Nvidia,BigCodeopendefault64.1%not datedQwen2.5-Coder Technical Report
99Qwen/Qwen-14B-ChatEpoch: Qwen-14B-Chat, AlibabaopenShots: 064.0%not datedYi: Open Foundation Models by 01.AI
100Yi-6B01.AIno page matchopenShots: 564.0%not datedStanford CRFM Leaderboard
101gemma-7bGoogle DeepMindno page matchopenShots: 563.6%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
102LLaMA-65BMeta AIno page matchopenShots: 563.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
103LLaMA-65BMeta AIno page matchopenShots: 563.4%not datedLLaMA: Open and Efficient Foundation Language Models
104Yi-6B01.AIno page matchopenShots: 563.2%not datedYi: Open Foundation Models by 01.AI
105Llama-2-34bMeta AIclosedShots: 562.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
106Llama-2-34bMeta AIclosedShots: 562.6%not datedQwen Technical Report
107Llama-2-34bMeta AIclosedShots: 562.6%not datedNemotron-4 15B Technical Report
108Qwen1.5-7BAlibabano page matchopenShots: 562.6%not datedStanford CRFM Leaderboard
109mistralai/Mistral-7B-Instruct-v0.2Epoch: Mistral-7B-Instruct-v0.2, Mistral AIopenShots: 562.5%not datedGemma: Open Models Based on Gemini Research and Technology
110mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 562.5%not datedMixtral of Experts
111mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 561.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
112Qwen2.5-Coder-32BAlibabano page matchopendefault61.2%not datedQwen2.5-Coder Technical Report
11301-ai/Yi-6B-ChatEpoch: Yi-6B-Chat, 01.AIopenShots: 561.0%not datedYi: Open Foundation Models by 01.AI
114deepseek-ai/DeepSeek-Coder-V2-Lite-BaseEpoch: DeepSeek-Coder-V2-Lite-Baseopendefault60.5%not datedQwen2.5-Coder Technical Report
115mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 560.1%not datedNemotron-4 15B Technical Report
116Gopher (280B)DeepMindclosedShots: 560.0%not datedScaling Language Models: Methods, Analysis & Insights from Training Gopher
117Gopher (280B)DeepMindclosedShots: 560.0%not datedLLaMA: Open and Efficient Foundation Language Models
118meta-llama/Llama-2-70b-chatEpoch: Llama-2-70b-chat, Meta AIopenShots: 559.9%not datedYi: Open Foundation Models by 01.AI
119mistralai/Mistral-7B-Instruct-v0.3Epoch: Mistral-7B-Instruct-v0.3, Mistral AIopenShots: 559.9%not datedStanford CRFM Leaderboard
120meta-llama/Llama-2-70b-chatEpoch: Llama-2-70b-chat, Meta AIopenShots: 059.4%not datedYi: Open Foundation Models by 01.AI
121Baichuan-2-13B-BaseBaichuanno page matchopenShots: 559.2%not datedNemotron-4 15B Technical Report
122Baichuan-2-13B-BaseBaichuanno page matchopenShots: 559.2%not datedBaichuan 2: Open Large-scale Language Models
123LLaMA-33BMeta AIno page matchopenShots: 558.7%not datedQwen Technical Report
124Nemotron-4 15BNvidiaclosedShots: 558.7%not datedNemotron-4 15B Technical Report
125tiiuae/falcon-11BEpoch: falcon-11b, Technology Innovation InstituteopenShots: 558.4%not datedFalcon2-11B Technical Report
126microsoft/phi-2Epoch: phi-2, MicrosoftopenShots: 558.4%not datedStanford CRFM Leaderboard
12701-ai/Yi-6B-ChatEpoch: Yi-6B-Chat, 01.AIopenShots: 058.2%not datedYi: Open Foundation Models by 01.AI
128LLaMA-33BMeta AIno page matchopenShots: 557.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
129LLaMA-33BMeta AIno page matchopenShots: 557.8%not datedLLaMA: Open and Efficient Foundation Language Models
130internlm/internlm-chat-20bEpoch: internlm-chat-20bopenShots: 557.4%not datedYi: Open Foundation Models by 01.AI
131falcon-40bTechnology Innovation Instituteno page matchopenShots: 556.9%not datedFalcon2-11B Technical ReportFalcon2-11B Technical Report
132LLaMA-33BMeta AIno page matchopenShots: 556.8%not datedMixtral of Experts
133mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 556.6%not datedStanford CRFM Leaderboard
134meta-llama/Llama-3.2-11B-Vision-InstructEpoch: Llama-3.2-11B-Vision-Instruct, Meta AIopenShots: 556.5%not datedStanford CRFM Leaderboard
135microsoft/phi-2Epoch: phi-2, MicrosoftopenShots: 556.3%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
136meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopenShots: 556.1%not datedStanford CRFM Leaderboard
137internlm/internlm-chat-20bEpoch: internlm-chat-20bopenShots: 055.6%not datedYi: Open Foundation Models by 01.AI
138Llama-2-13bMeta AIno page matchopenShots: 555.6%not datedMixtral of Experts
139falcon-40bTechnology Innovation Instituteno page matchopenShots: 555.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
140falcon-40bTechnology Innovation Instituteno page matchopenShots: 555.4%not datedQwen Technical Report
141Llama-2-13bMeta AIno page matchopenShots: 555.4%not datedStanford CRFM Leaderboard
142baichuan-inc/Baichuan2-13B-ChatEpoch: Baichuan2-13B-ChatopenShots: 055.1%not datedYi: Open Foundation Models by 01.AI
143Llama-2-13bMeta AIno page matchopenShots: 555.1%not datedBaichuan 2: Open Large-scale Language Models
144Llama-2-13bMeta AIno page matchopenShots: 554.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
145Llama-2-13bMeta AIno page matchopenShots: 554.8%not datedGemma: Open Models Based on Gemini Research and Technology
146Llama-2-13bMeta AIno page matchopenShots: 554.8%not datedNemotron-4 15B Technical Report
147Baichuan-2-7B-BaseBaichuanno page matchopenShots: 554.2%not datedBaichuan 2: Open Large-scale Language Models
148PaLM 62Bnot statedShots: 553.7%not datedLLaMA: Open and Efficient Foundation Language Models
149Qwen2.5-Coder-1.5BAlibabano page matchopendefault53.6%not datedQwen2.5-Coder Technical Report
150Qwen-14BAlibabano page matchopenShots: 553.4%not datedQwen Technical Report
151baichuan-inc/Baichuan-13B-BaseEpoch: Baichuan-13B-Base, BaichuanopenShots: 551.6%not datedBaichuan 2: Open Large-scale Language Models
152internlm/internlm-7bEpoch: internlm-7bopenShots: 551.0%not datedQwen Technical Report
153meta-llama/Llama-2-13b-chatEpoch: Llama-2-13b-chat, Meta AIopenShots: 050.9%not datedYi: Open Foundation Models by 01.AI
154baichuan-inc/Baichuan2-13B-ChatEpoch: Baichuan2-13B-ChatopenShots: 550.1%not datedYi: Open Foundation Models by 01.AI
155INTELLECT-1-InstructPrime Intellect,Hugging Face,Arcee AInot statedShots: 549.9%not datedINTELLECT-1 Technical Report
156zai-org/chatglm2-6bEpoch: chatglm2-6bopenShots: 547.9%not datedQwen Technical Report
157mpt-30bMosaicMLno page matchopenShots: 547.9%not datedQwen Technical Report
158LLaMA-13BMeta AIno page matchopenShots: 547.7%not datedQwen Technical Report
159meta-llama/Llama-2-13b-chatEpoch: Llama-2-13b-chat, Meta AIopenShots: 547.3%not datedYi: Open Foundation Models by 01.AI
160LLaMA-13BMeta AIno page matchopenShots: 546.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
161LLaMA-13BMeta AIno page matchopenShots: 546.9%not datedLLaMA: Open and Efficient Foundation Language Models
162mpt-30bMosaicMLno page matchopenShots: 546.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
163LLaMA-13BMeta AIno page matchopenShots: 546.3%not datedBaichuan 2: Open Large-scale Language Models
164Llama-2-7bMeta AIno page matchopenShots: 545.8%not datedStanford CRFM Leaderboard
165Llama-2-7bMeta AIno page matchopenShots: 545.7%not datedBaichuan 2: Open Large-scale Language Models
166Llama-2-7bMeta AIno page matchopenShots: 545.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
167Llama-2-7bMeta AIno page matchopenShots: 545.3%not datedGemma: Open Models Based on Gemini Research and Technology
168Llama-2-7bMeta AIno page matchopenShots: 245.3%not datedTextbooks Are All You Need II: phi-1.5 technical report
169Qwen-7BAlibabano page matchopenShots: 545.0%not datedQwen Technical Report
170Llama-2-7bMeta AIno page matchopenShots: 544.4%not datedMixtral of Experts
171text-davinci-001OpenAIclosedShots: 543.9%not datedScaling Language Models: Methods, Analysis & Insights from Training Gopher
172text-davinci-001OpenAIclosedShots: 543.9%not datedLLaMA: Open and Efficient Foundation Language Models
173baichuan-inc/Baichuan-7BEpoch: Baichuan-7B, BaichuanopenShots: 542.3%not datedBaichuan 2: Open Large-scale Language Models
174gemma-2bGoogle DeepMindno page matchopenShots: 542.3%not datedGemma: Open Models Based on Gemini Research and Technology
175Qwen/Qwen2.5-Coder-0.5BEpoch: Qwen2.5-Coder-0.5Bopendefault42.0%not datedQwen2.5-Coder Technical Report
176Qwen/CodeQwen1.5-7BEpoch: CodeQwen1.5-7Bopendefault40.5%not datedQwen2.5-Coder Technical Report
177deepseek-ai/deepseek-coder-33b-baseEpoch: deepseek-coder-33b-base, DeepSeek,Peking Universityopendefault39.4%not datedQwen2.5-Coder Technical Report
178bigcode/starcoder2-7bEpoch: starcoder2-7b, Hugging Face,ServiceNow,Nvidia,BigCodeopendefault38.8%not datedQwen2.5-Coder Technical Report
179microsoft/phi-1_5Epoch: phi-1_5, MicrosoftopenShots: 237.6%not datedTextbooks Are All You Need II: phi-1.5 technical report
180bigcode/starcoder2-3bEpoch: starcoder2-3b, Hugging Face,ServiceNow,Nvidia,BigCodeopendefault36.6%not datedQwen2.5-Coder Technical Report
181deepseek-ai/deepseek-coder-6.7b-baseEpoch: deepseek-coder-6.7b-base, DeepSeek,Peking Universityopendefault36.4%not datedQwen2.5-Coder Technical Report
182Salesforce/xgen-7b-8k-baseEpoch: xgen-7b-8k-base, SalesforceopenShots: 536.3%not datedXGen-7B Technical Report
183LLaMA-7BMeta AIno page matchopenShots: 535.6%not datedQwen Technical Report
184LLaMA-7BMeta AIno page matchopenShots: 235.2%not datedTextbooks Are All You Need II: phi-1.5 technical report
185LLaMA-7BMeta AIno page matchopenShots: 535.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
186LLaMA-7BMeta AIno page matchopenShots: 535.1%not datedLLaMA: Open and Efficient Foundation Language Models
187LLaMA-7BMeta AIno page matchopenShots: 535.1%not datedBaichuan 2: Open Large-scale Language Models
188LLaMA-7BMeta AIno page matchopenShots: 535.1%not datedXGen-7B Technical Report
189falcon-7bTechnology Innovation Instituteno page matchopenShots: 535.0%not datedFalcon2-11B Technical Report
190Salesforce/xgen-7b-8k-baseEpoch: xgen-7b-8k-base, SalesforceopenShots: 032.1%not datedXGen-7B Technical Report
191LLaMA-7BMeta AIno page matchopenShots: 032.0%not datedXGen-7B Technical Report
192mpt-7bMosaicMLno page matchopenShots: 530.8%not datedQwen Technical Report
193open_llama_7bnot statedShots: 529.9%not datedXGen-7B Technical Report
194open_llama_7bnot statedShots: 028.6%not datedXGen-7B Technical Report
195Qwen/Qwen-1_8BEpoch: Qwen-1_8BopenShots: 528.2%not datedQwen Technical Report
196mpt-7bMosaicMLno page matchopenShots: 527.9%not datedBaichuan 2: Open Large-scale Language Models
197mpt-7bMosaicMLno page matchopenShots: 027.4%not datedXGen-7B Technical Report
198falcon-7bTechnology Innovation Instituteno page matchopenShots: 226.9%not datedTextbooks Are All You Need II: phi-1.5 technical report
199falcon-7bTechnology Innovation Instituteno page matchopenShots: 526.9%not datedXGen-7B Technical Report
200mpt-7bMosaicMLno page matchopenShots: 526.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
201mpt-7bMosaicMLno page matchopenShots: 226.8%not datedTextbooks Are All You Need II: phi-1.5 technical report
202mpt-7bMosaicMLno page matchopenShots: 526.7%not datedXGen-7B Technical Report
203RedPajama-INCITE-7B-Basenot statedShots: 526.3%not datedXGen-7B Technical Report
204Cerebras-GPT-13BCerebras Systemsno page matchopenShots: 526.2%not datedXGen-7B Technical Report
205dolly-v2-12bDatabricksno page matchopenShots: 526.2%not datedXGen-7B Technical Report
206falcon-7bTechnology Innovation Instituteno page matchopenShots: 526.2%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
207falcon-7bTechnology Innovation Instituteno page matchopenShots: 526.2%not datedQwen Technical Report
208falcon-7bTechnology Innovation Instituteno page matchopenShots: 526.0%not datedBaichuan 2: Open Large-scale Language Models
209deepseek-ai/deepseek-coder-1.3b-baseEpoch: deepseek-coder-1.3b-base, DeepSeek,Peking Universityopendefault25.8%not datedQwen2.5-Coder Technical Report
210RedPajama-INCITE-7B-Basenot statedShots: 025.8%not datedXGen-7B Technical Report
211gpt-j-6bEleutherAI,LAIONno page matchopenShots: 025.7%not datedXGen-7B Technical Report
212dolly-v2-12bDatabricksno page matchopenShots: 025.4%not datedXGen-7B Technical Report
213gpt-j-6bEleutherAI,LAIONno page matchopenShots: 525.1%not datedXGen-7B Technical Report
214facebook/opt-13bEpoch: opt-13bopenShots: 525.1%not datedXGen-7B Technical Report
215Cerebras-GPT-13BCerebras Systemsno page matchopenShots: 024.6%not datedXGen-7B Technical Report
216facebook/opt-13bEpoch: opt-13bopenShots: 024.4%not datedXGen-7B Technical Report
217falcon-7bTechnology Innovation Instituteno page matchopenShots: 023.9%not datedXGen-7B Technical Report

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method