TriviaQA
ReportedTrivia questions, answered without being given the source text.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Score shown
- Epoch's column "EM", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
- Results
- 115 in all: 78 for open-weight models, 34 for closed ones, 3 where Epoch does not say. 14 linked to a model page here.
Best open-weight result: meta-llama/Llama-2-70b-hf, 87.6%, number 1 of 115 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 5 | 87.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 2 | claude-2.0Anthropic | closed | Shots: 5 | 87.5% | not dated | Model Card and Evaluations for Claude Models |
| 3 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 64 | 87.5% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 4 | claude-1.3Anthropic | closed | Shots: 5 | 86.7% | not dated | Model Card and Evaluations for Claude Models |
| 5 | PaLM 2-L | not stated | Shots: 1 | 86.1% | not dated | PaLM 2 Technical Report |
| 6 | LLaMA-65BMeta AIno page match | open | Shots: 64 | 86.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 7 | LLaMA-65BMeta AIno page match | open | Shots: 5 | 85.9% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 8 | gpt-3.5-turbo-1106OpenAI | closed | Shots: 5 | 85.8% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 9 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 1 | 85.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 10 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 1 | 85.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 11 | gpt-4-0613OpenAI | closed | default | 84.8% | not dated | PapersWithCode |
| 12 | Llama-2-34bMeta AI | closed | Shots: 64 | 84.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 13 | Llama-2-34bMeta AI | closed | Shots: 5 | 84.5% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 14 | LLaMA-65BMeta AIno page match | open | Shots: 1 | 84.5% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 15 | LLaMA-33BMeta AIno page match | open | Shots: 5 | 83.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 16 | LLaMA-33BMeta AIno page match | open | Shots: 64 | 83.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 17 | Llama-2-34bMeta AI | closed | Shots: 1 | 83.3% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 18 | deepseek-ai/DeepSeek-V3Epoch: DeepSeek-V3, DeepSeek | open | Shots: 5 | 82.9% | not dated | DeepSeek-V3 Technical Report |
| 19 | Llama-3.1-405BMeta AIno page match | open | Shots: 5 | 82.7% | not dated | DeepSeek-V3 Technical Report |
| 20 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 0 | 82.4% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 21 | mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AI | open | Shots: 5 | 82.2% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 22 | LLaMA-65BMeta AIno page match | open | Shots: 0 | 81.7% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 23 | PaLM 2-M | not stated | Shots: 1 | 81.7% | not dated | PaLM 2 Technical Report |
| 24 | PaLM 540BGoogle Research | closed | Shots: 1 | 81.4% | not dated | PaLM: Scaling Language Modeling with Pathways |
| 25 | PaLM 540BGoogle Research | closed | Shots: few | 81.4% | not dated | PaLM: Scaling Language Modeling with Pathways |
| 26 | Llama-2-34bMeta AI | closed | Shots: 0 | 81.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 27 | LLaMA-33BMeta AIno page match | open | Shots: 1 | 80.7% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 28 | DeepSeek-V2DeepSeekno page match | open | Shots: 5 | 80.0% | not dated | DeepSeek-V3 Technical Report |
| 29 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 5 | 79.9% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 30 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 64 | 79.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 31 | Llama-2-13bMeta AIno page match | open | Shots: 5 | 79.6% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 32 | Llama-2-13bMeta AIno page match | open | Shots: 5 | 79.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 33 | Llama-2-13bMeta AIno page match | open | Shots: 64 | 79.4% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 34 | claude-instant-1.1Anthropic | closed | Shots: 5 | 78.9% | not dated | Model Card and Evaluations for Claude Models |
| 35 | claude-instant-1.1Anthropic | closed | Shots: 5 | 78.9% | not dated | Releasing Claude Instant 1.2 |
| 36 | claude-instant-1.2Anthropic | closed | Shots: 5 | 78.7% | not dated | Releasing Claude Instant 1.2 |
| 37 | LLaMA-33BMeta AIno page match | open | Shots: 0 | 78.7% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 38 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 1 | 78.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 39 | LLaMA-13BMeta AIno page match | open | Shots: 64 | 77.9% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 40 | Llama-2-13bMeta AIno page match | open | Shots: 1 | 77.2% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 41 | LLaMA-13BMeta AIno page match | open | Shots: 5 | 77.1% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 42 | PaLM 540BGoogle Research | closed | Shots: 0 | 76.9% | not dated | PaLM: Scaling Language Modeling with Pathways |
| 43 | GLaM (MoE)Google | closed | Shots: 1 | 75.8% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 44 | GLaM (MoE)Google | closed | Shots: 1 | 75.8% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 45 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | Shots: 5 | 75.2% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 46 | PaLM 2-S | not stated | Shots: 1 | 75.2% | not dated | PaLM 2 Technical Report |
| 47 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 0 | 74.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 48 | LLaMA-13BMeta AIno page match | open | Shots: 1 | 74.4% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 49 | microsoft/Phi-3-medium-128k-instructEpoch: Phi-3-medium-128k-instruct, Microsoft | open | Shots: 5 | 73.9% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 50 | Llama-2-7bMeta AIno page match | open | Shots: 64 | 73.7% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 51 | mpt-30bMosaicMLno page match | open | Shots: 64 | 73.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 52 | mpt-30bMosaicMLno page match | open | Shots: 5 | 73.3% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 53 | Llama-2-13bMeta AIno page match | open | Shots: 0 | 73.1% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 54 | LLaMA-65BMeta AIno page match | open | Shots: 64 | 73.0% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 55 | LLaMA-65BMeta AIno page match | open | Shots: 5 | 72.6% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 56 | gemma-7bGoogle DeepMindno page match | open | Shots: 5 | 72.3% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 57 | Llama-2-7bMeta AIno page match | open | Shots: 5 | 72.1% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 58 | Llama-2-7bMeta AIno page match | open | Shots: 5 | 72.1% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 59 | Qwen2.5-72BAlibabano page match | open | Shots: 5 | 71.9% | not dated | DeepSeek-V3 Technical Report |
| 60 | LLaMA-65BMeta AIno page match | open | Shots: 1 | 71.6% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 61 | GLaM (MoE)Google | closed | Shots: 0 | 71.3% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 62 | mpt-30bMosaicMLno page match | open | Shots: 1 | 71.3% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 63 | text-davinci-001OpenAI | closed | Shots: few | 71.2% | not dated | Language Models are Few-Shot Learners |
| 64 | text-davinci-001OpenAI | closed | Shots: 64 | 71.2% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 65 | text-davinci-001OpenAI | closed | Shots: few | 71.2% | not dated | Language Models are Few-Shot Learners |
| 66 | LLaMA-7BMeta AIno page match | open | Shots: 64 | 71.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 67 | LLaMA-33BMeta AIno page match | open | Shots: 64 | 70.4% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 68 | LLaMA-7BMeta AIno page match | open | Shots: 5 | 70.4% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 69 | LLaMA-13BMeta AIno page match | open | Shots: 0 | 70.1% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 70 | LLaMA-33BMeta AIno page match | open | Shots: 5 | 69.9% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 71 | Llama-2-7bMeta AIno page match | open | Shots: 1 | 68.9% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 72 | LLaMA-65BMeta AIno page match | open | Shots: 0 | 68.2% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 73 | mpt-30bMosaicMLno page match | open | Shots: 0 | 68.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 74 | text-davinci-001OpenAI | closed | Shots: 1 | 68.0% | not dated | Language Models are Few-Shot Learners |
| 75 | text-davinci-001OpenAI | closed | Shots: 1 | 68.0% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 76 | text-davinci-001OpenAI | closed | Shots: 1 | 68.0% | not dated | Language Models are Few-Shot Learners |
| 77 | LLaMA-33BMeta AIno page match | open | Shots: 1 | 67.9% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 78 | meta-llama/Meta-Llama-3-8B-InstructEpoch: Meta-Llama-3-8B-Instruct, Meta AI | open | Shots: 5 | 67.7% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 79 | LLaMA-7BMeta AIno page match | open | Shots: 1 | 67.4% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 80 | Llama-2-7bMeta AIno page match | open | Shots: 0 | 65.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 81 | LLaMA-33BMeta AIno page match | open | Shots: 0 | 65.1% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 82 | Chinchilla (70B)DeepMind | closed | Shots: 64 | 64.6% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 83 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 5 | 64.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 84 | text-davinci-001OpenAI | closed | Shots: 0 | 64.3% | not dated | Language Models are Few-Shot Learners |
| 85 | text-davinci-001OpenAI | closed | Shots: 0 | 64.3% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 86 | text-davinci-001OpenAI | closed | Shots: 0 | 64.3% | not dated | Language Models are Few-Shot Learners |
| 87 | Chinchilla (70B)DeepMind | closed | Shots: 5 | 64.1% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 88 | LLaMA-13BMeta AIno page match | open | Shots: 64 | 64.0% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 89 | microsoft/Phi-3-mini-4k-instructEpoch: Phi-3-mini-4k-instruct, Microsoft | open | Shots: 5 | 64.0% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 90 | gemma-7bGoogle DeepMindno page match | open | Shots: 5 | 63.4% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 91 | LLaMA-7BMeta AIno page match | open | Shots: 0 | 63.3% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 92 | LLaMA-13BMeta AIno page match | open | Shots: 5 | 63.1% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 93 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | Shots: 5 | 62.5% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 94 | mpt-7bMosaicMLno page match | open | Shots: 64 | 61.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 95 | mpt-7bMosaicMLno page match | open | Shots: 5 | 61.2% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 96 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 64 | 61.1% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 97 | LLaMA-13BMeta AIno page match | open | Shots: 1 | 60.5% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 98 | mpt-7bMosaicMLno page match | open | Shots: 1 | 59.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 99 | microsoft/Phi-3-small-8k-instructEpoch: Phi-3-small-8k-instruct, Microsoft | open | Shots: 5 | 58.1% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 100 | LLaMA-7BMeta AIno page match | open | Shots: 64 | 57.6% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 101 | Gopher (280B)DeepMind | closed | Shots: 64 | 57.2% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 102 | Gopher (280B)DeepMind | closed | Shots: 64 | 57.1% | not dated | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts |
| 103 | Gopher (280B)DeepMind | closed | Shots: 64 | 57.1% | not dated | Scaling Language Models: Methods, Analysis & Insights from Training Gopher |
| 104 | Gopher (280B)DeepMind | closed | Shots: 5 | 57.0% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 105 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 1 | 56.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 106 | LLaMA-13BMeta AIno page match | open | Shots: 0 | 56.6% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 107 | LLaMA-7BMeta AIno page match | open | Shots: 5 | 56.3% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 108 | mpt-7bMosaicMLno page match | open | Shots: 0 | 55.7% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 109 | Chinchilla (70B)DeepMind | closed | Shots: 0 | 55.4% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 110 | LLaMA-7BMeta AIno page match | open | Shots: 1 | 53.4% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 111 | gemma-2bGoogle DeepMindno page match | open | Shots: 5 | 53.2% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 112 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 0 | 52.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 113 | LLaMA-7BMeta AIno page match | open | Shots: 0 | 50.0% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 114 | microsoft/phi-2Epoch: phi-2, Microsoft | open | Shots: 5 | 45.2% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 115 | Gopher (280B)DeepMind | closed | Shots: 0 | 43.5% | not dated | LLaMA: Open and Efficient Foundation Language Models |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method