Skip to content
Weights Not Walls
Menu

BoolQ

Reported

Yes or no questions about a short passage.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Score shown
Epoch's column "Score", as a percentage, higher is better. The source gives no error bars.
Results
136 in all: 88 for open-weight models, 33 for closed ones, 15 where Epoch does not say. 24 linked to a model page here.

Best open-weight result: T5-11B, 91.2%, number 1 of 136 results.

BoolQ results
#ModelWeightsSettingScoreDateSource
1T5-11BGoogleno page matchopendefault91.2%not datedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
2PaLM 2-Lnot statedShots: 190.9%not datedThe Falcon Series of Open Language Models
3T5-3BGoogleno page matchopendefault89.9%not datedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
4Inflection-1Inflection AIcloseddefault89.7%not datedThe Falcon Series of Open Language Models
5stabilityai/StableBeluga2Epoch: StableBeluga2, Stability AIopendefault89.4%not datedQwen Technical Report
6falcon-180BTechnology Innovation Instituteno page matchopenShots: 189.0%not datedThe Falcon Series of Open Language Models
7gpt-4o-mini-2024-07-18OpenAIclosedShots: 288.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
8PaLM 540BGoogle ResearchclosedShots: 188.7%not datedThe Falcon Series of Open Language Models
9meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 588.6%not datedStanford HELM
10PaLM 2-Mnot statedShots: 188.6%not datedThe Falcon Series of Open Language Models
11PaLM 2-Snot statedShots: 188.1%not datedThe Falcon Series of Open Language Models
12text-davinci-003OpenAInot statedShots: 588.1%not datedStanford HELM
13PaLM 540BGoogle ResearchclosedShots: 088.0%not datedLLaMA: Open and Efficient Foundation Language Models
14PaLM 540BGoogle Researchcloseddefault88.0%not datedThe Falcon Series of Open Language Models
15falcon-180BTechnology Innovation Instituteno page matchopendefault87.8%not datedThe Falcon Series of Open Language Models
16meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopendefault87.7%not datedQwen Technical Report
17text-davinci-002OpenAIclosedShots: 587.7%not datedStanford HELM
18internlm/internlm-20bEpoch: internlm-20bopendefault87.5%not datedQwen Technical Report
19mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 587.4%not datedStanford HELM
20LLaMA-65BMeta AIno page matchopenShots: 587.1%not datedStanford HELM
21gpt-3.5-turbo-0613OpenAIclosedShots: 587.0%not datedStanford HELM
22LLaMA-65BMeta AIno page matchopendefault86.6%not datedQwen Technical Report
23Qwen-14BAlibabano page matchopendefault86.2%not datedQwen Technical Report
24LLaMA-33BMeta AIno page matchopenShots: 586.1%not datedStanford HELM
25gemini-1.5-flash-001Google DeepMindclosedShots: 285.8%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
26gemma-2-9bGoogle DeepMindno page matchopenShots: 285.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
27T5-Largenot stateddefault85.4%not datedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
28LLaMA-65BMeta AIno page matchopendefault85.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
29LLaMA-65BMeta AIno page matchopenShots: 085.3%not datedLLaMA: Open and Efficient Foundation Language Models
30meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopendefault85.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
31meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopendefault85.0%not datedThe Falcon Series of Open Language Models
32mpt-30b-instructMosaicMLno page matchopenShots: 585.0%not datedStanford HELM
33Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: few84.8%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
34PaLM 62Bnot statedShots: 084.8%not datedLLaMA: Open and Efficient Foundation Language Models
35microsoft/Phi-3.5-MoE-instructEpoch: Phi-3.5-MoE-instruct, MicrosoftopenShots: 284.6%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
36LLaMA-33BMeta AIno page matchopendefault84.4%not datedQwen Technical Report
37Chinchilla (70B)DeepMindclosedShots: 083.7%not datedLLaMA: Open and Efficient Foundation Language Models
38Chinchilla (70B)DeepMindcloseddefault83.7%not datedThe Falcon Series of Open Language Models
39Chinchilla (70B)DeepMindclosedShots: 083.7%not datedTraining Compute-Optimal Large Language Models
40Llama-2-34bMeta AIcloseddefault83.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
41Llama-2-34bMeta AIcloseddefault83.7%not datedThe Falcon Series of Open Language Models
42vicuna-13b-v1.1not statedShots: 083.5%not datedTextbooks Are All You Need II: phi-1.5 technical report
43gemma-7bGoogle DeepMindno page matchopenShots: 083.2%not datedGemma: Open Models Based on Gemini Research and Technology
44mistralai/Mistral-7B-Instruct-v0.2Epoch: Mistral-7B-Instruct-v0.2, Mistral AIopenShots: 083.2%not datedGemma: Open Models Based on Gemini Research and Technology
45falcon-40bTechnology Innovation Instituteno page matchopendefault83.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
46LLaMA-33BMeta AIno page matchopendefault83.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
47LLaMA-33BMeta AIno page matchopenShots: 083.1%not datedLLaMA: Open and Efficient Foundation Language Models
48meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AIopenShots: 282.8%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
49Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 182.5%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
50mistralai/Mistral-Nemo-Base-2407Epoch: Mistral-Nemo-Base-2407, Mistral AIopenShots: 282.5%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
51Llama-2-13bMeta AIno page matchopendefault82.4%not datedQwen Technical Report
52falcon-40bTechnology Innovation Instituteno page matchopenShots: 581.9%not datedStanford HELM
53falcon-40bTechnology Innovation Instituteno page matchopendefault81.9%not datedThe Falcon Series of Open Language Models
54Llama-2-13bMeta AIno page matchopenShots: 081.7%not datedGemma: Open Models Based on Gemini Research and Technology
55Llama-2-13bMeta AIno page matchopendefault81.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
56Llama-2-13bMeta AIno page matchopendefault81.7%not datedThe Falcon Series of Open Language Models
57T5-Basenot stateddefault81.4%not datedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
58Llama-2-13bMeta AIno page matchopenShots: 581.1%not datedStanford HELM
59vicuna-13b-v1.3Large Model Systems Organization,University of California (UC) Berkeleyno page matchopenShots: 580.8%not datedStanford HELM
60mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 280.5%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
61Gopher (280B)DeepMindcloseddefault79.4%not datedThe Falcon Series of Open Language Models
62Gopher (280B)DeepMindclosedShots: 079.3%not datedLLaMA: Open and Efficient Foundation Language Models
63Gopher (280B)DeepMindclosedShots: 079.3%not datedTraining Compute-Optimal Large Language Models
64opt-175bMeta AIno page matchopenShots: 579.3%not datedStanford HELM
65zai-org/chatglm2-6bEpoch: chatglm2-6bopendefault79.0%not datedQwen Technical Report
66mpt-30bMosaicMLno page matchopendefault79.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
67LLaMA-13BMeta AIno page matchopendefault78.7%not datedQwen Technical Report
68Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 078.2%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
69Megatron-Turing NLG 530BMicrosoft,Nvidiacloseddefault78.2%not datedThe Falcon Series of Open Language Models
70Megatron-Turing NLG 530BMicrosoft,NvidiaclosedShots: 078.2%not datedTraining Compute-Optimal Large Language Models
71LLaMA-13BMeta AIno page matchopendefault78.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
72LLaMA-13BMeta AIno page matchopenShots: 078.1%not datedLLaMA: Open and Efficient Foundation Language Models
73microsoft/Phi-3.5-mini-instructEpoch: Phi-3.5-mini-instruct, MicrosoftopenShots: 278.0%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
74Llama-2-7bMeta AIno page matchopenShots: 077.9%not datedTextbooks Are All You Need II: phi-1.5 technical report
75text-davinci-001OpenAIclosedShots: few77.5%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
76text-davinci-001OpenAIclosedShots: few77.5%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
77Llama-2-7bMeta AIno page matchopendefault77.4%not datedQwen Technical Report
78Llama-2-7bMeta AIno page matchopenShots: 077.4%not datedGemma: Open Models Based on Gemini Research and Technology
79Llama-2-7bMeta AIno page matchopendefault77.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
80Llama-2-7bMeta AIno page matchopendefault77.4%not datedThe Falcon Series of Open Language Models
81text-davinci-001OpenAIclosedShots: 176.7%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
82text-davinci-001OpenAIclosedShots: 176.7%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
83LLaMA-7BMeta AIno page matchopendefault76.5%not datedQwen Technical Report
84LLaMA-7BMeta AIno page matchopendefault76.5%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
85LLaMA-7BMeta AIno page matchopenShots: 076.5%not datedLLaMA: Open and Efficient Foundation Language Models
86Qwen-7BAlibabano page matchopendefault76.4%not datedQwen Technical Report
87T5-Smallnot stateddefault76.4%not datedExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
88text-davinci-001OpenAIclosedShots: few76.4%not datedLanguage Models are Few-Shot Learners
89Llama-2-7bMeta AIno page matchopenShots: 576.2%not datedStanford HELM
90T5-11BGoogleno page matchopendefault76.1%not datedStanford HELM
91facebook/opt-66bEpoch: opt-66b, Meta AIopenShots: 576.0%not datedStanford HELM
92microsoft/phi-1_5Epoch: phi-1_5, MicrosoftopenShots: 075.8%not datedTextbooks Are All You Need II: phi-1.5 technical report
93LLaMA-7BMeta AIno page matchopenShots: 575.6%not datedStanford HELM
94falcon-7bTechnology Innovation Instituteno page matchopenShots: 575.3%not datedStanford HELM
95mpt-7bMosaicMLno page matchopendefault75.0%not datedQwen Technical Report
96mpt-7bMosaicMLno page matchopendefault75.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
97LLaMA-7BMeta AIno page matchopenShots: 074.9%not datedXGen-7B Technical Report
98Salesforce/xgen-7b-8k-baseEpoch: xgen-7b-8k-base, SalesforceopenShots: 074.3%not datedXGen-7B Technical Report
99mpt-7bMosaicMLno page matchopenShots: 074.1%not datedXGen-7B Technical Report
100mpt-7bMosaicMLno page matchopenShots: 073.9%not datedTextbooks Are All You Need II: phi-1.5 technical report
101falcon-7bTechnology Innovation Instituteno page matchopendefault73.8%not datedThe Falcon Series of Open Language Models
102falcon-7bTechnology Innovation Instituteno page matchopenShots: 073.8%not datedXGen-7B Technical Report
103LLaMA-7BMeta AIno page matchopenShots: 073.2%not datedTextbooks Are All You Need II: phi-1.5 technical report
104davinciOpenAIclosedShots: 572.2%not datedStanford HELM
105LLaMA-13BMeta AIno page matchopenShots: 571.4%not datedStanford HELM
106open_llama_7bnot statedShots: 070.6%not datedXGen-7B Technical Report
107bigscience/bloomEpoch: bloom, Hugging Face,BigScienceopenShots: 570.4%not datedStanford HELM
108mpt-30bMosaicMLno page matchopenShots: 570.4%not datedStanford HELM
109gemma-2bGoogle DeepMindno page matchopenShots: 069.4%not datedGemma: Open Models Based on Gemini Research and Technology
110RedPajama-INCITE-7B-Basenot statedShots: 069.3%not datedXGen-7B Technical Report
111falcon-7bTechnology Innovation Instituteno page matchopenShots: 068.5%not datedTextbooks Are All You Need II: phi-1.5 technical report
112Qwen/Qwen-1_8BEpoch: Qwen-1_8Bopendefault68.0%not datedQwen Technical Report
113falcon-7bTechnology Innovation Instituteno page matchopendefault67.5%not datedQwen Technical Report
114falcon-7bTechnology Innovation Instituteno page matchopendefault67.5%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
115Baichuan-2-13B-BaseBaichuanno page matchopendefault67.0%not datedQwen Technical Report
116curieOpenAInot statedShots: 565.6%not datedStanford HELM
117gpt-j-6bEleutherAI,LAIONno page matchopenShots: 065.4%not datedXGen-7B Technical Report
118facebook/opt-13bEpoch: opt-13bopenShots: 065.0%not datedXGen-7B Technical Report
119EleutherAI/gpt-neox-20bEpoch: gpt-neox-20b, EleutherAIopenShots: 064.9%not datedXGen-7B Technical Report
120internlm/internlm-7bEpoch: internlm-7bopendefault64.1%not datedQwen Technical Report
121Baichuan-2-7B-BaseBaichuanno page matchopendefault63.2%not datedQwen Technical Report
122text-curie-001OpenAIclosedShots: 562.0%not datedStanford HELM
123gpt-neo-2.7BEleutherAIno page matchopenShots: 061.8%not datedTextbooks Are All You Need II: phi-1.5 technical report
124gpt2-xlOpenAIno page matchopenShots: 061.8%not datedTextbooks Are All You Need II: phi-1.5 technical report
125Cerebras-GPT-13BCerebras Systemsno page matchopenShots: 061.1%not datedXGen-7B Technical Report
126text-davinci-001OpenAIclosedShots: 060.5%not datedUsing DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
127text-davinci-001OpenAIclosedShots: 060.5%not datedLLaMA: Open and Efficient Foundation Language Models
128text-davinci-001OpenAIcloseddefault60.5%not datedThe Falcon Series of Open Language Models
129text-davinci-001OpenAIclosedShots: 060.5%not datedTraining Compute-Optimal Large Language Models
130facebook/opt-1.3bEpoch: opt-1.3b, Meta AIopenShots: 059.6%not datedTextbooks Are All You Need II: phi-1.5 technical report
131stabilityai/stablelm-tuned-alpha-7bEpoch: stablelm-tuned-alpha-7bopenShots: 059.0%not datedXGen-7B Technical Report
132adaOpenAInot statedShots: 558.1%not datedStanford HELM
133babbageOpenAInot statedShots: 557.4%not datedStanford HELM
134dolly-v2-12bDatabricksno page matchopenShots: 056.3%not datedXGen-7B Technical Report
135text-ada-001OpenAInot statedShots: 546.4%not datedStanford HELM
136text-babbage-001OpenAIclosedShots: 545.1%not datedStanford HELM

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method