Skip to content
Weights Not Walls
Menu

PIQA

Reported

Physical common sense: choosing the better of two ways to do an everyday task.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Score shown
Epoch's column "Score", as a percentage, higher is better; random guessing scores about 50.0%. The source gives no error bars.
Results
113 in all: 82 for open-weight models, 27 for closed ones, 4 where Epoch does not say. 25 linked to a model page here.

Best open-weight result: microsoft/Phi-3.5-MoE-instruct, 88.6%, number 2 of 113 results.

PIQA results
#ModelWeightsSettingScoreDateSource
1gpt-4o-mini-2024-07-18OpenAIclosedShots: 588.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
2microsoft/Phi-3.5-MoE-instructEpoch: Phi-3.5-MoE-instruct, MicrosoftopenShots: 588.6%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
3gemini-1.5-flash-002Google DeepMindclosedShots: 587.5%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
4Llama-3.1-405BMeta AIno page matchopenShots: 085.9%not datedDeepSeek-V3 Technical Report
5falcon-180BTechnology Innovation Instituteno page matchopendefault84.9%not datedThe Falcon Series of Open Language Models
6deepseek-ai/DeepSeek-V3-BaseEpoch: DeepSeek-V3-Base, DeepSeekopenShots: 084.7%not datedDeepSeek-V3 Technical Report
7Inflection-1Inflection AIcloseddefault84.2%not datedThe Falcon Series of Open Language Models
8DeepSeek-V2DeepSeekno page matchopenShots: 083.9%not datedDeepSeek-V3 Technical Report
9gemma-2-9bGoogle DeepMindno page matchopenShots: 583.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
10mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AIopendefault83.6%not datedMixtral of Experts
11mistralai/Mistral-Nemo-Base-2407Epoch: Mistral-Nemo-Base-2407, Mistral AIopenShots: 583.5%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
12stabilityai/StableBeluga2Epoch: StableBeluga2, Stability AIopenShots: 083.3%not datedQwen Technical Report
13Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: few83.2%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
14falcon-40bTechnology Innovation Instituteno page matchopendefault83.0%not datedThe Falcon Series of Open Language Models
15mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopendefault83.0%not datedNemotron-4 15B Technical Report
16meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 082.8%not datedQwen Technical Report
17meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopendefault82.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
18meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopendefault82.8%not datedThe Falcon Series of Open Language Models
19LLaMA-65BMeta AIno page matchopenShots: 082.8%not datedQwen Technical Report
20LLaMA-65BMeta AIno page matchopendefault82.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
21LLaMA-65BMeta AIno page matchopendefault82.8%not datedLLaMA: Open and Efficient Foundation Language Models
22meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopendefault82.6%not datedMixtral of Experts
23Qwen2.5-72BAlibabano page matchopenShots: 082.6%not datedDeepSeek-V3 Technical Report
24falcon-40bTechnology Innovation Instituteno page matchopendefault82.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
25Nemotron-4 15BNvidiacloseddefault82.4%not datedNemotron-4 15B Technical Report
26LLaMA-33BMeta AIno page matchopenShots: 082.3%not datedQwen Technical Report
27LLaMA-33BMeta AIno page matchopendefault82.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
28LLaMA-33BMeta AIno page matchopendefault82.3%not datedLLaMA: Open and Efficient Foundation Language Models
29PaLM 540BGoogle Researchcloseddefault82.3%not datedLLaMA: Open and Efficient Foundation Language Models
30PaLM 540BGoogle Researchcloseddefault82.3%not datedThe Falcon Series of Open Language Models
31text-davinci-001OpenAIclosedShots: few82.3%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
32text-davinci-001OpenAIclosedShots: few82.3%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
33LLaMA-33BMeta AIno page matchopendefault82.2%not datedMixtral of Experts
34mistralai/Mistral-7B-Instruct-v0.1Epoch: Mistral-7B-Instruct-v0.1, Mistral AIopendefault82.2%not datedMixtral of Experts
35mistralai/Mistral-7B-Instruct-v0.2Epoch: Mistral-7B-Instruct-v0.2, Mistral AIopenShots: 082.2%not datedGemma: Open Models Based on Gemini Research and Technology
36Megatron-Turing NLG 530BMicrosoft,Nvidiacloseddefault82.0%not datedThe Falcon Series of Open Language Models
37Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 082.0%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
38Llama-2-34bMeta AIcloseddefault81.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
39Llama-2-34bMeta AIcloseddefault81.9%not datedNemotron-4 15B Technical Report
40Llama-2-34bMeta AIcloseddefault81.9%not datedThe Falcon Series of Open Language Models
41mpt-30bMosaicMLno page matchopendefault81.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
42Chinchilla (70B)DeepMindcloseddefault81.8%not datedLLaMA: Open and Efficient Foundation Language Models
43Chinchilla (70B)DeepMindcloseddefault81.8%not datedThe Falcon Series of Open Language Models
44Gopher (280B)DeepMindclosedShots: 081.8%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
45Gopher (280B)DeepMindclosedShots: 081.8%not datedScaling Language Models: Methods, Analysis & Insights from Training Gopher
46Gopher (280B)DeepMindcloseddefault81.8%not datedLLaMA: Open and Efficient Foundation Language Models
47Gopher (280B)DeepMindcloseddefault81.8%not datedThe Falcon Series of Open Language Models
48gemma-7bGoogle DeepMindno page matchopenShots: 081.2%not datedGemma: Open Models Based on Gemini Research and Technology
49gemma-7bGoogle DeepMindno page matchopendefault81.2%not datedNemotron-4 15B Technical Report
50meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopenShots: 581.2%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
51microsoft/Phi-3.5-mini-instructEpoch: Phi-3.5-mini-instruct, MicrosoftopenShots: 581.0%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
52text-davinci-001OpenAIclosedShots: 081.0%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
53text-davinci-001OpenAIcloseddefault81.0%not datedLLaMA: Open and Efficient Foundation Language Models
54text-davinci-001OpenAIcloseddefault81.0%not datedThe Falcon Series of Open Language Models
55Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 181.0%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
56Llama-2-13bMeta AIno page matchopendefault80.8%not datedMixtral of Experts
57mpt-7bMosaicMLno page matchopenShots: 080.6%not datedQwen Technical Report
58mpt-7bMosaicMLno page matchopendefault80.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
59Llama-2-13bMeta AIno page matchopenShots: 080.5%not datedGemma: Open Models Based on Gemini Research and Technology
60Llama-2-13bMeta AIno page matchopenShots: 080.5%not datedQwen Technical Report
61Llama-2-13bMeta AIno page matchopendefault80.5%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
62Llama-2-13bMeta AIno page matchopendefault80.5%not datedThe Falcon Series of Open Language Models
63PaLM 62Bnot stateddefault80.5%not datedLLaMA: Open and Efficient Foundation Language Models
64text-davinci-001OpenAIclosedShots: 180.5%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
65text-davinci-001OpenAIclosedShots: 180.5%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
66text-davinci-001OpenAIclosedShots: 080.4%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
67falcon-7bTechnology Innovation Instituteno page matchopendefault80.3%not datedThe Falcon Series of Open Language Models
68internlm/internlm-20bEpoch: internlm-20bopenShots: 080.3%not datedQwen Technical Report
69LLaMA-13BMeta AIno page matchopenShots: 080.1%not datedQwen Technical Report
70LLaMA-13BMeta AIno page matchopendefault80.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
71LLaMA-13BMeta AIno page matchopendefault80.1%not datedLLaMA: Open and Efficient Foundation Language Models
72Qwen-14BAlibabano page matchopenShots: 079.9%not datedQwen Technical Report
73Qwen-14BAlibabano page matchopendefault79.9%not datedNemotron-4 15B Technical Report
74Llama-2-13bMeta AIno page matchopendefault79.8%not datedNemotron-4 15B Technical Report
75LLaMA-7BMeta AIno page matchopenShots: 079.8%not datedQwen Technical Report
76LLaMA-7BMeta AIno page matchopendefault79.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
77LLaMA-7BMeta AIno page matchopendefault79.8%not datedLLaMA: Open and Efficient Foundation Language Models
78falcon-7bTechnology Innovation Instituteno page matchopendefault79.4%not datedTextbooks Are All You Need II: phi-1.5 technical report
79falcon-7bTechnology Innovation Instituteno page matchopenShots: 079.4%not datedXGen-7B Technical Report
80mpt-7bMosaicMLno page matchopenShots: 079.1%not datedXGen-7B Technical Report
81mpt-7bMosaicMLno page matchopendefault78.9%not datedTextbooks Are All You Need II: phi-1.5 technical report
82Llama-2-7bMeta AIno page matchopenShots: 078.8%not datedGemma: Open Models Based on Gemini Research and Technology
83Llama-2-7bMeta AIno page matchopenShots: 078.8%not datedQwen Technical Report
84Llama-2-7bMeta AIno page matchopendefault78.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
85Llama-2-7bMeta AIno page matchopendefault78.8%not datedThe Falcon Series of Open Language Models
86LLaMA-7BMeta AIno page matchopenShots: 078.7%not datedXGen-7B Technical Report
87Baichuan-2-13B-BaseBaichuanno page matchopenShots: 078.1%not datedQwen Technical Report
88Baichuan-2-13B-BaseBaichuanno page matchopendefault78.1%not datedNemotron-4 15B Technical Report
89Llama-2-7bMeta AIno page matchopendefault78.1%not datedTextbooks Are All You Need II: phi-1.5 technical report
90internlm/internlm-7bEpoch: internlm-7bopenShots: 077.9%not datedQwen Technical Report
91Llama-2-7bMeta AIno page matchopendefault77.9%not datedMixtral of Experts
92LLaMA-7BMeta AIno page matchopendefault77.9%not datedTextbooks Are All You Need II: phi-1.5 technical report
93Qwen-7BAlibabano page matchopenShots: 077.9%not datedQwen Technical Report
94vicuna-13b-v1.1not stateddefault77.4%not datedTextbooks Are All You Need II: phi-1.5 technical report
95gemma-2bGoogle DeepMindno page matchopenShots: 077.3%not datedGemma: Open Models Based on Gemini Research and Technology
96RedPajama-INCITE-7B-Basenot statedShots: 076.9%not datedXGen-7B Technical Report
97falcon-7bTechnology Innovation Instituteno page matchopenShots: 076.7%not datedQwen Technical Report
98falcon-7bTechnology Innovation Instituteno page matchopendefault76.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
99EleutherAI/gpt-neox-20bEpoch: gpt-neox-20b, EleutherAIopenShots: 076.7%not datedXGen-7B Technical Report
100baichuan-inc/Baichuan-7BEpoch: Baichuan-7B, BaichuanopenShots: 076.2%not datedQwen Technical Report
101open_llama_7bnot statedShots: 076.0%not datedXGen-7B Technical Report
102facebook/opt-13bEpoch: opt-13bopenShots: 075.7%not datedXGen-7B Technical Report
103Salesforce/xgen-7b-8k-baseEpoch: xgen-7b-8k-base, SalesforceopenShots: 075.5%not datedXGen-7B Technical Report
104dolly-v2-12bDatabricksno page matchopenShots: 075.4%not datedXGen-7B Technical Report
105gpt-j-6bEleutherAI,LAIONno page matchopenShots: 075.4%not datedXGen-7B Technical Report
106Cerebras-GPT-13BCerebras Systemsno page matchopenShots: 073.5%not datedXGen-7B Technical Report
107mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 573.4%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
108Qwen/Qwen-1_8BEpoch: Qwen-1_8BopenShots: 073.3%not datedQwen Technical Report
109gpt-neo-2.7BEleutherAIno page matchopendefault72.9%not datedTextbooks Are All You Need II: phi-1.5 technical report
110gpt2-xlOpenAIno page matchopendefault70.5%not datedTextbooks Are All You Need II: phi-1.5 technical report
111zai-org/chatglm2-6bEpoch: chatglm2-6bopenShots: 069.6%not datedQwen Technical Report
112facebook/opt-1.3bEpoch: opt-1.3b, Meta AIopendefault69.0%not datedTextbooks Are All You Need II: phi-1.5 technical report
113stabilityai/stablelm-tuned-alpha-7bEpoch: stablelm-tuned-alpha-7bopenShots: 065.8%not datedXGen-7B Technical Report

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method