Skip to content
Weights Not Walls
Menu

TriviaQA

Reported

Trivia questions, answered without being given the source text.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Score shown
Epoch's column "EM", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
Results
115 in all: 78 for open-weight models, 34 for closed ones, 3 where Epoch does not say. 14 linked to a model page here.

Best open-weight result: meta-llama/Llama-2-70b-hf, 87.6%, number 1 of 115 results.

TriviaQA results
#ModelWeightsSettingScoreDateSource
1meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 587.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
2claude-2.0AnthropicclosedShots: 587.5%not datedModel Card and Evaluations for Claude Models
3meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 6487.5%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
4claude-1.3AnthropicclosedShots: 586.7%not datedModel Card and Evaluations for Claude Models
5PaLM 2-Lnot statedShots: 186.1%not datedPaLM 2 Technical Report
6LLaMA-65BMeta AIno page matchopenShots: 6486.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
7LLaMA-65BMeta AIno page matchopenShots: 585.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
8gpt-3.5-turbo-1106OpenAIclosedShots: 585.8%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
9meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 185.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
10meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 185.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
11gpt-4-0613OpenAIcloseddefault84.8%not datedPapersWithCode
12Llama-2-34bMeta AIclosedShots: 6484.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
13Llama-2-34bMeta AIclosedShots: 584.5%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
14LLaMA-65BMeta AIno page matchopenShots: 184.5%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
15LLaMA-33BMeta AIno page matchopenShots: 583.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
16LLaMA-33BMeta AIno page matchopenShots: 6483.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
17Llama-2-34bMeta AIclosedShots: 183.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
18deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeekopenShots: 582.9%not datedDeepSeek-V3 Technical Report
19Llama-3.1-405BMeta AIno page matchopenShots: 582.7%not datedDeepSeek-V3 Technical Report
20meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AIopenShots: 082.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
21mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AIopenShots: 582.2%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
22LLaMA-65BMeta AIno page matchopenShots: 081.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
23PaLM 2-Mnot statedShots: 181.7%not datedPaLM 2 Technical Report
24PaLM 540BGoogle ResearchclosedShots: 181.4%not datedPaLM: Scaling Language Modeling with Pathways
25PaLM 540BGoogle ResearchclosedShots: few81.4%not datedPaLM: Scaling Language Modeling with Pathways
26Llama-2-34bMeta AIclosedShots: 081.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
27LLaMA-33BMeta AIno page matchopenShots: 180.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
28DeepSeek-V2DeepSeekno page matchopenShots: 580.0%not datedDeepSeek-V3 Technical Report
29falcon-40bTechnology Innovation Instituteno page matchopenShots: 579.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
30falcon-40bTechnology Innovation Instituteno page matchopenShots: 6479.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
31Llama-2-13bMeta AIno page matchopenShots: 579.6%not datedGemma: Open Models Based on Gemini Research and Technology
32Llama-2-13bMeta AIno page matchopenShots: 579.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
33Llama-2-13bMeta AIno page matchopenShots: 6479.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
34claude-instant-1.1AnthropicclosedShots: 578.9%not datedModel Card and Evaluations for Claude Models
35claude-instant-1.1AnthropicclosedShots: 578.9%not datedReleasing Claude Instant 1.2
36claude-instant-1.2AnthropicclosedShots: 578.7%not datedReleasing Claude Instant 1.2
37LLaMA-33BMeta AIno page matchopenShots: 078.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
38falcon-40bTechnology Innovation Instituteno page matchopenShots: 178.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
39LLaMA-13BMeta AIno page matchopenShots: 6477.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
40Llama-2-13bMeta AIno page matchopenShots: 177.2%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
41LLaMA-13BMeta AIno page matchopenShots: 577.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
42PaLM 540BGoogle ResearchclosedShots: 076.9%not datedPaLM: Scaling Language Modeling with Pathways
43GLaM (MoE)GoogleclosedShots: 175.8%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
44GLaM (MoE)GoogleclosedShots: 175.8%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
45mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 575.2%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
46PaLM 2-Snot statedShots: 175.2%not datedPaLM 2 Technical Report
47falcon-40bTechnology Innovation Instituteno page matchopenShots: 074.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
48LLaMA-13BMeta AIno page matchopenShots: 174.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
49microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, MicrosoftopenShots: 573.9%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
50Llama-2-7bMeta AIno page matchopenShots: 6473.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
51mpt-30bMosaicMLno page matchopenShots: 6473.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
52mpt-30bMosaicMLno page matchopenShots: 573.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
53Llama-2-13bMeta AIno page matchopenShots: 073.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
54LLaMA-65BMeta AIno page matchopenShots: 6473.0%not datedLLaMA: Open and Efficient Foundation Language Models
55LLaMA-65BMeta AIno page matchopenShots: 572.6%not datedLLaMA: Open and Efficient Foundation Language Models
56gemma-7bGoogle DeepMindno page matchopenShots: 572.3%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
57Llama-2-7bMeta AIno page matchopenShots: 572.1%not datedGemma: Open Models Based on Gemini Research and Technology
58Llama-2-7bMeta AIno page matchopenShots: 572.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
59Qwen2.5-72BAlibabano page matchopenShots: 571.9%not datedDeepSeek-V3 Technical Report
60LLaMA-65BMeta AIno page matchopenShots: 171.6%not datedLLaMA: Open and Efficient Foundation Language Models
61GLaM (MoE)GoogleclosedShots: 071.3%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
62mpt-30bMosaicMLno page matchopenShots: 171.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
63text-davinci-001OpenAIclosedShots: few71.2%not datedLanguage Models are Few-Shot Learners
64text-davinci-001OpenAIclosedShots: 6471.2%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
65text-davinci-001OpenAIclosedShots: few71.2%not datedLanguage Models are Few-Shot Learners
66LLaMA-7BMeta AIno page matchopenShots: 6471.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
67LLaMA-33BMeta AIno page matchopenShots: 6470.4%not datedLLaMA: Open and Efficient Foundation Language Models
68LLaMA-7BMeta AIno page matchopenShots: 570.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
69LLaMA-13BMeta AIno page matchopenShots: 070.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
70LLaMA-33BMeta AIno page matchopenShots: 569.9%not datedLLaMA: Open and Efficient Foundation Language Models
71Llama-2-7bMeta AIno page matchopenShots: 168.9%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
72LLaMA-65BMeta AIno page matchopenShots: 068.2%not datedLLaMA: Open and Efficient Foundation Language Models
73mpt-30bMosaicMLno page matchopenShots: 068.0%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
74text-davinci-001OpenAIclosedShots: 168.0%not datedLanguage Models are Few-Shot Learners
75text-davinci-001OpenAIclosedShots: 168.0%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
76text-davinci-001OpenAIclosedShots: 168.0%not datedLanguage Models are Few-Shot Learners
77LLaMA-33BMeta AIno page matchopenShots: 167.9%not datedLLaMA: Open and Efficient Foundation Language Models
78meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AIopenShots: 567.7%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
79LLaMA-7BMeta AIno page matchopenShots: 167.4%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
80Llama-2-7bMeta AIno page matchopenShots: 065.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
81LLaMA-33BMeta AIno page matchopenShots: 065.1%not datedLLaMA: Open and Efficient Foundation Language Models
82Chinchilla (70B)DeepMindclosedShots: 6464.6%not datedLLaMA: Open and Efficient Foundation Language Models
83falcon-7bTechnology Innovation Instituteno page matchopenShots: 564.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
84text-davinci-001OpenAIclosedShots: 064.3%not datedLanguage Models are Few-Shot Learners
85text-davinci-001OpenAIclosedShots: 064.3%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
86text-davinci-001OpenAIclosedShots: 064.3%not datedLanguage Models are Few-Shot Learners
87Chinchilla (70B)DeepMindclosedShots: 564.1%not datedLLaMA: Open and Efficient Foundation Language Models
88LLaMA-13BMeta AIno page matchopenShots: 6464.0%not datedLLaMA: Open and Efficient Foundation Language Models
89microsoft/Phi-3-mini-4k-instructEpoch: Phi-3-mini-4k-instruct, MicrosoftopenShots: 564.0%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
90gemma-7bGoogle DeepMindno page matchopenShots: 563.4%not datedGemma: Open Models Based on Gemini Research and Technology
91LLaMA-7BMeta AIno page matchopenShots: 063.3%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
92LLaMA-13BMeta AIno page matchopenShots: 563.1%not datedLLaMA: Open and Efficient Foundation Language Models
93mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AIopenShots: 562.5%not datedGemma: Open Models Based on Gemini Research and Technology
94mpt-7bMosaicMLno page matchopenShots: 6461.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
95mpt-7bMosaicMLno page matchopenShots: 561.2%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
96falcon-7bTechnology Innovation Instituteno page matchopenShots: 6461.1%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
97LLaMA-13BMeta AIno page matchopenShots: 160.5%not datedLLaMA: Open and Efficient Foundation Language Models
98mpt-7bMosaicMLno page matchopenShots: 159.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
99microsoft/Phi-3-small-8k-instructEpoch: Phi-3-small-8k-instruct, MicrosoftopenShots: 558.1%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
100LLaMA-7BMeta AIno page matchopenShots: 6457.6%not datedLLaMA: Open and Efficient Foundation Language Models
101Gopher (280B)DeepMindclosedShots: 6457.2%not datedLLaMA: Open and Efficient Foundation Language Models
102Gopher (280B)DeepMindclosedShots: 6457.1%not datedGLaM: Efficient Scaling of Language Models with Mixture-of-Experts
103Gopher (280B)DeepMindclosedShots: 6457.1%not datedScaling Language Models: Methods, Analysis & Insights from Training Gopher
104Gopher (280B)DeepMindclosedShots: 557.0%not datedLLaMA: Open and Efficient Foundation Language Models
105falcon-7bTechnology Innovation Instituteno page matchopenShots: 156.8%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
106LLaMA-13BMeta AIno page matchopenShots: 056.6%not datedLLaMA: Open and Efficient Foundation Language Models
107LLaMA-7BMeta AIno page matchopenShots: 556.3%not datedLLaMA: Open and Efficient Foundation Language Models
108mpt-7bMosaicMLno page matchopenShots: 055.7%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
109Chinchilla (70B)DeepMindclosedShots: 055.4%not datedLLaMA: Open and Efficient Foundation Language Models
110LLaMA-7BMeta AIno page matchopenShots: 153.4%not datedLLaMA: Open and Efficient Foundation Language Models
111gemma-2bGoogle DeepMindno page matchopenShots: 553.2%not datedGemma: Open Models Based on Gemini Research and Technology
112falcon-7bTechnology Innovation Instituteno page matchopenShots: 052.6%not datedLlama 2: Open Foundation and Fine-Tuned Chat Models
113LLaMA-7BMeta AIno page matchopenShots: 050.0%not datedLLaMA: Open and Efficient Foundation Language Models
114microsoft/phi-2Epoch: phi-2, MicrosoftopenShots: 545.2%not datedPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
115Gopher (280B)DeepMindclosedShots: 043.5%not datedLLaMA: Open and Efficient Foundation Language Models

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method