Skip to content
Weights Not Walls
Menu

HellaSwag

Reported

Choosing the most sensible way to continue a short description of an everyday situation.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Score shown
Epoch's column "Overall accuracy", as a percentage, higher is better; random guessing scores about 25.0%. The source gives no error bars.
Results
113 in all: 71 for open-weight models, 28 for closed ones, 14 where Epoch does not say. 29 linked to a model page here.

Best open-weight result: Llama-3.1-405B, 89.2%, number 3 of 113 results.

HellaSwag results
#ModelWeightsSettingScoreDateSource
1gpt-4-0314OpenAIclosedShots: 1095.3%not datedGPT-4 Technical Report
2gpt-4-32k-0314OpenAIclosedShots: 1095.3%not datedThe Falcon Series of Open Language Models
3Llama-3.1-405BMeta AIno page matchopenShots: 1089.2%not datedDeepSeek-V3 Technical Report
4falcon-180BTechnology Innovation Instituteno page matchopenShots: 1089.0%not datedThe Falcon Series of Open Language Models
5deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopenShots: 1088.9%not datedDeepSeek-V3 Technical Report
6DeepSeek-V2DeepSeekno page matchopenShots: 1087.1%not datedDeepSeek-V3 Technical Report
7PaLM 2-Lnot statedShots: 186.8%not datedPaLM 2 Technical Report
8mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AIopenShots: 1086.7%not datedMixtral of Experts
9text-davinci-003OpenAInot statedShots: 1085.5%not datedThe Falcon Series of Open Language Models
10text-davinci-003OpenAInot statedShots: 1085.5%not datedGPT-4 Technical Report
11meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 085.3%not datedQwen Technical Report
12falcon-40bTechnology Innovation Instituteno page matchopenShots: 1085.3%not datedFalcon2-11B Technical Report
13falcon-40bTechnology Innovation Instituteno page matchopenShots: 1085.3%not datedFalcon2-11B Technical Report
14Qwen2.5-72BAlibabano page matchopenShots: 1084.8%not datedDeepSeek-V3 Technical Report
15mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AIopendefault84.4%not datedMixtral of Experts
16LLaMA-65BMeta AIno page matchopenShots: 084.2%not datedLLaMA: Open and Efficient Foundation Language Models
17stabilityai/StableBeluga2Epoch: StableBeluga2, Stability AIopenShots: 084.1%not datedQwen Technical Report
18PaLM 2-Mnot statedShots: 184.0%not datedPaLM 2 Technical Report
19PaLM 540BGoogle ResearchclosedShots: few83.8%not datedPaLM: Scaling Language Modeling with Pathways
20PaLM 540BGoogle ResearchclosedShots: 183.6%not datedPaLM: Scaling Language Modeling with Pathways
21PaLM 540BGoogle ResearchclosedShots: 083.4%not datedPaLM: Scaling Language Modeling with Pathways
22Qwen2.5-Coder-32BAlibabano page matchopendefault83.0%not datedQwen2.5-Coder Technical Report
23tiiuae/falcon-11BEpoch: falcon-11b, Technology Innovation InstituteopenShots: 1082.9%not datedFalcon2-11B Technical Report
24tiiuae/falcon-11BEpoch: falcon-11b, Technology Innovation InstituteopenShots: 1082.9%not datedFalcon2-11B Technical Report
25falcon-40bTechnology Innovation Instituteno page matchopenShots: 082.8%not datedFalcon2-11B Technical Report
26falcon-40bTechnology Innovation Instituteno page matchopenShots: 082.8%not datedFalcon2-11B Technical Report
27LLaMA-33BMeta AIno page matchopenShots: 082.8%not datedLLaMA: Open and Efficient Foundation Language Models
28Megatron-Turing NLG 530BMicrosoft,Nvidiacloseddefault82.4%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
29Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: few82.4%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
30Nemotron-4 15BNvidiaclosedShots: 082.4%not datedNemotron-4 15B Technical Report
31microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, MicrosoftopenShots: 582.4%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
32gemma-7bGoogle DeepMindno page matchopendefault82.2%not datedGemma: Open Models Based on Gemini Research and Technology
33text-davinci-003OpenAInot statedShots: 082.2%not datedStanford HELM
34tiiuae/falcon-11BEpoch: falcon-11b, Technology Innovation InstituteopenShots: 082.1%not datedFalcon2-11B Technical Report
35tiiuae/falcon-11BEpoch: falcon-11b, Technology Innovation InstituteopenShots: 082.1%not datedFalcon2-11B Technical Report
36PaLM 2-Snot statedShots: 182.0%not datedPaLM 2 Technical Report
37text-davinci-002OpenAIclosedShots: 081.5%not datedStanford HELM
38gemma-7bGoogle DeepMindno page matchopenShots: 081.2%not datedGemma: Open Models Based on Gemini Research and Technology
39mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopendefault81.0%not datedMixtral of Experts
40Llama-2-13bMeta AIno page matchopenShots: 080.7%not datedGemma: Open Models Based on Gemini Research and Technology
41Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 080.2%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
42Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 180.2%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
43Qwen/Qwen2.5-Coder-14BEpoch: Qwen2.5-Coder-14Bopendefault80.2%not datedQwen2.5-Coder Technical Report
44text-davinci-001OpenAIclosedShots: few79.3%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
45text-davinci-001OpenAIclosedShots: 2079.3%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
46text-davinci-001OpenAIclosedShots: few79.3%not datedLanguage Models are Few-Shot Learners
47Gopher (280B)DeepMindclosedShots: 079.2%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
48Gopher (280B)DeepMindclosedShots: 079.2%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
49LLaMA-13BMeta AIno page matchopenShots: 079.2%not datedLLaMA: Open and Efficient Foundation Language Models
50opt-175bMeta AIno page matchopenShots: 079.1%not datedStanford HELM
51text-davinci-001OpenAIclosedShots: 078.9%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
52text-davinci-001OpenAIclosedShots: 078.9%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
53text-davinci-001OpenAIclosedShots: 078.9%not datedLanguage Models are Few-Shot Learners
54falcon-7bTechnology Innovation Instituteno page matchopenShots: 1078.1%not datedFalcon2-11B Technical Report
55falcon-7bTechnology Innovation Instituteno page matchopenShots: 1078.1%not datedFalcon2-11B Technical Report
56internlm/internlm-20bEpoch: internlm-20bopenShots: 078.1%not datedQwen Technical Report
57text-davinci-001OpenAIclosedShots: 178.1%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
58text-davinci-001OpenAIclosedShots: 178.1%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
59text-davinci-001OpenAIclosedShots: 178.1%not datedLanguage Models are Few-Shot Learners
60davinciOpenAIclosedShots: 077.5%not datedStanford HELM
61GLaM (MoE)GoogleclosedShots: 877.2%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
62Llama-2-7bMeta AIno page matchopenShots: 077.2%not datedGemma: Open Models Based on Gemini Research and Technology
63microsoft/Phi-3-small-8k-instructEpoch: Phi-3-small-8k-instruct, MicrosoftopenShots: 577.0%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
64GLaM (MoE)GoogleclosedShots: 176.8%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
65Qwen2.5-Coder-7BAlibabano page matchopendefault76.8%not datedQwen2.5-Coder Technical Report
66microsoft/Phi-3-mini-4k-instructEpoch: Phi-3-mini-4k-instruct, MicrosoftopenShots: 576.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
67GLaM (MoE)GoogleclosedShots: 076.6%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
68falcon-7bTechnology Innovation Instituteno page matchopenShots: 076.4%not datedXGen-7B Technical Report
69mpt-7bMosaicMLno page matchopenShots: 076.4%not datedQwen Technical Report
7001-ai/Yi-9BEpoch: Yi-9Bopendefault76.4%not datedYi: Open Foundation Models by 01.AI
71falcon-7bTechnology Innovation Instituteno page matchopenShots: 076.3%not datedFalcon2-11B Technical Report
72falcon-7bTechnology Innovation Instituteno page matchopenShots: 076.3%not datedFalcon2-11B Technical Report
73LLaMA-7BMeta AIno page matchopenShots: 076.2%not datedXGen-7B Technical Report
74LLaMA-7BMeta AIno page matchopenShots: 076.1%not datedLLaMA: Open and Efficient Foundation Language Models
75mpt-7bMosaicMLno page matchopenShots: 076.1%not datedXGen-7B Technical Report
76facebook/opt-66bEpoch: opt-66b, Meta AIopenShots: 074.5%not datedStanford HELM
77bigscience/bloomEpoch: bloom, Hugging Face,BigScienceopenShots: 074.4%not datedStanford HELM
78Yi-6B01.AIno page matchopenShots: 1074.4%not datedYi: Open Foundation Models by 01.AI
79Salesforce/xgen-7b-8k-baseEpoch: xgen-7b-8k-base, SalesforceopenShots: 074.2%not datedXGen-7B Technical Report
80falcon-7bTechnology Innovation Instituteno page matchopenShots: 074.1%not datedQwen Technical Report
81open_llama_7bnot statedShots: 071.8%not datedXGen-7B Technical Report
82INTELLECT-1-InstructPrime Intellect,Hugging Face,Arcee AInot statedShots: 1071.4%not datedINTELLECT-1 Technical Report
83gemma-2bGoogle DeepMindno page matchopenShots: 071.4%not datedGemma: Open Models Based on Gemini Research and Technology
84Qwen/Qwen2.5-Coder-3BEpoch: Qwen2.5-Coder-3Bopendefault70.9%not datedQwen2.5-Coder Technical Report
85Baichuan-2-13B-BaseBaichuanno page matchopenShots: 070.8%not datedQwen Technical Report
86dolly-v2-12bDatabricksno page matchopenShots: 070.8%not datedXGen-7B Technical Report
87internlm/internlm-7bEpoch: internlm-7bopenShots: 070.6%not datedQwen Technical Report
88EleutherAI/gpt-neox-20bEpoch: gpt-neox-20b, EleutherAIopenShots: 070.5%not datedXGen-7B Technical Report
89RedPajama-INCITE-7B-Basenot statedShots: 070.3%not datedXGen-7B Technical Report
90facebook/opt-13bEpoch: opt-13bopenShots: 069.9%not datedXGen-7B Technical Report
91curieOpenAInot statedShots: 068.2%not datedStanford HELM
92Baichuan-2-7B-BaseBaichuanno page matchopenShots: 068.0%not datedQwen Technical Report
93text-curie-001OpenAIclosedShots: 067.6%not datedStanford HELM
94gpt-j-6bEleutherAI,LAIONno page matchopenShots: 066.2%not datedXGen-7B Technical Report
95Qwen2.5-Coder-1.5BAlibabano page matchopendefault61.8%not datedQwen2.5-Coder Technical Report
96Cerebras-GPT-13BCerebras Systemsno page matchopenShots: 059.4%not datedXGen-7B Technical Report
97vicuna-13b-v1.1not stateddefault57.8%not datedTextbooks Are All You Need II: phi-1.5 technical report
98Llama-2-7bMeta AIno page matchopendefault57.1%not datedTextbooks Are All You Need II: phi-1.5 technical report
99mpt-7bMosaicMLno page matchopendefault57.1%not datedTextbooks Are All You Need II: phi-1.5 technical report
100zai-org/chatglm2-6bEpoch: chatglm2-6bopenShots: 057.0%not datedQwen Technical Report
101LLaMA-7BMeta AIno page matchopendefault56.2%not datedTextbooks Are All You Need II: phi-1.5 technical report
102text-babbage-001OpenAIclosedShots: 056.1%not datedStanford HELM
103babbageOpenAInot statedShots: 055.5%not datedStanford HELM
104falcon-7bTechnology Innovation Instituteno page matchopendefault54.2%not datedTextbooks Are All You Need II: phi-1.5 technical report
105microsoft/phi-2Epoch: phi-2, MicrosoftopenShots: 553.6%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
106Qwen/Qwen2.5-Coder-0.5BEpoch: Qwen2.5-Coder-0.5Bopendefault48.4%not datedQwen2.5-Coder Technical Report
107microsoft/phi-1_5Epoch: phi-1_5, MicrosoftopenShots: 047.6%not datedTextbooks Are All You Need II: phi-1.5 technical report
108adaOpenAInot statedShots: 043.5%not datedStanford HELM
109text-ada-001OpenAInot statedShots: 042.9%not datedStanford HELM
110gpt-neo-2.7BEleutherAIno page matchopendefault42.7%not datedTextbooks Are All You Need II: phi-1.5 technical report
111facebook/opt-1.3bEpoch: opt-1.3b, Meta AIopendefault41.5%not datedTextbooks Are All You Need II: phi-1.5 technical report
112stabilityai/stablelm-tuned-alpha-7bEpoch: stablelm-tuned-alpha-7bopenShots: 040.7%not datedXGen-7B Technical Report
113gpt2-xlOpenAIno page matchopendefault40.0%not datedTextbooks Are All You Need II: phi-1.5 technical report

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method