GSM8K
ReportedGrade-school mathematics word problems that need a few steps of arithmetic.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Score shown
- Epoch's column "EM", as a percentage, higher is better; random guessing scores about 0.0%. The source gives no error bars.
- Results
- 170 in all: 140 for open-weight models, 19 for closed ones, 11 where Epoch does not say. 63 linked to a model page here.
Best open-weight result: deepseek-ai/DeepSeek-Coder-V2-Instruct, 94.5%, number 1 of 170 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | deepseek-ai/DeepSeek-Coder-V2-InstructEpoch: DeepSeek-Coder-V2-Instruct, DeepSeek | open | default | 94.5% | not dated | Qwen2.5-Coder Technical Report |
| 2 | Qwen/Qwen2.5-Coder-14B-InstructEpoch: Qwen2.5-Coder-14B-Instruct | open | default | 94.2% | not dated | Qwen2.5-Coder Technical Report |
| 3 | Qwen/Qwen2.5-Coder-32B-InstructEpoch: Qwen2.5-Coder-32B-Instruct, Alibaba | open | default | 93.0% | not dated | Qwen2.5-Coder Technical Report |
| 4 | gpt-4-0314OpenAI | closed | Shots: 5 | 92.0% | not dated | GPT-4 Technical Report |
| 5 | gpt-4o-mini-2024-07-18OpenAI | closed | Shots: 8 | 91.3% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 6 | Qwen2.5-Coder-32BAlibabano page match | open | Shots: 4 | 91.1% | not dated | Qwen2.5-Coder Technical Report |
| 7 | gpt-4-0613OpenAI | closed | Shots: 4 | 90.0% | not dated | Baichuan 2: Open Large-scale Language Models |
| 8 | microsoft/Phi-3.5-MoE-instructEpoch: Phi-3.5-MoE-instruct, Microsoft | open | Shots: 8 | 88.7% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 9 | Qwen/Qwen2.5-Coder-14BEpoch: Qwen2.5-Coder-14B | open | Shots: 4 | 88.7% | not dated | Qwen2.5-Coder Technical Report |
| 10 | deepseek-ai/DeepSeek-Coder-V2-Lite-InstructEpoch: DeepSeek-Coder-V2-Lite-Instruct | open | default | 87.6% | not dated | Qwen2.5-Coder Technical Report |
| 11 | claude-instant-1.2Anthropic | closed | Shots: 0 | 86.7% | not dated | Releasing Claude Instant 1.2 |
| 12 | Qwen/Qwen2.5-Coder-7B-InstructEpoch: Qwen2.5-Coder-7B-Instruct, Alibaba | open | default | 86.7% | not dated | Qwen2.5-Coder Technical Report |
| 13 | microsoft/Phi-3.5-mini-instructEpoch: Phi-3.5-mini-instruct, Microsoft | open | Shots: 8 | 86.2% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 14 | deepseek-ai/DeepSeek-Coder-V2-BaseEpoch: DeepSeek-Coder-V2-Base, DeepSeek | open | Shots: 4 | 85.8% | not dated | Qwen2.5-Coder Technical Report |
| 15 | gemma-2-9bGoogle DeepMindno page match | open | Shots: 8 | 84.9% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 16 | mistralai/Mistral-Nemo-Base-2407Epoch: Mistral-Nemo-Base-2407, Mistral AI | open | Shots: 8 | 84.2% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 17 | Qwen2.5-Coder-7BAlibabano page match | open | Shots: 4 | 83.9% | not dated | Qwen2.5-Coder Technical Report |
| 18 | gemini-1.5-flash-001Google DeepMind | closed | Shots: 8 | 82.4% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 19 | meta-llama/Llama-3.1-8B-InstructEpoch: Llama-3.1-8B-Instruct, Meta AI | open | Shots: 8 | 82.4% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 20 | claude-instant-1.1Anthropic | closed | Shots: 0 | 80.9% | not dated | Releasing Claude Instant 1.2 |
| 21 | Qwen/Qwen2.5-Coder-3B-InstructEpoch: Qwen2.5-Coder-3B-Instruct | open | default | 80.7% | not dated | Qwen2.5-Coder Technical Report |
| 22 | text-davinci-003OpenAI | not stated | Shots: 8 | 78.2% | not dated | Qwen Technical Report |
| 23 | 01-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AI | open | Shots: 4 | 76.0% | not dated | Yi: Open Foundation Models by 01.AI |
| 24 | Qwen/Qwen2.5-Coder-3BEpoch: Qwen2.5-Coder-3B | open | Shots: 4 | 75.7% | not dated | Qwen2.5-Coder Technical Report |
| 25 | mistralai/Mixtral-8x7B-v0.1Epoch: Mixtral-8x7B-v0.1, Mistral AI | open | Shots: 8 | 74.4% | not dated | Mixtral of Experts |
| 26 | 01-ai/Yi-34B-ChatEpoch: Yi-34B-Chat, 01.AI | open | Shots: 0 | 71.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 27 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 8 | 69.6% | not dated | Mixtral of Experts |
| 28 | stabilityai/StableBeluga2Epoch: StableBeluga2, Stability AI | open | Shots: 8 | 69.6% | not dated | Qwen Technical Report |
| 29 | Yi-34B01.AIno page match | open | default | 67.2% | not dated | Yi: Open Foundation Models by 01.AI |
| 30 | deepseek-ai/DeepSeek-Coder-V2-Lite-BaseEpoch: DeepSeek-Coder-V2-Lite-Base | open | Shots: 4 | 67.1% | not dated | Qwen2.5-Coder Technical Report |
| 31 | Qwen2.5-Coder-1.5BAlibabano page match | open | Shots: 4 | 65.8% | not dated | Qwen2.5-Coder Technical Report |
| 32 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 8 | 63.3% | not dated | Qwen Technical Report |
| 33 | internlm/internlm-20bEpoch: internlm-20b | open | default | 62.9% | not dated | Yi: Open Foundation Models by 01.AI |
| 34 | Qwen-14BAlibabano page match | open | default | 61.3% | not dated | Yi: Open Foundation Models by 01.AI |
| 35 | Qwen-14BAlibabano page match | open | Shots: 8 | 61.3% | not dated | Qwen Technical Report |
| 36 | Qwen/Qwen-14B-ChatEpoch: Qwen-14B-Chat, Alibaba | open | Shots: 4 | 61.2% | not dated | Yi: Open Foundation Models by 01.AI |
| 37 | Qwen-14BAlibabano page match | open | Shots: 8 | 60.1% | not dated | Nemotron-4 15B Technical Report |
| 38 | Qwen/Qwen-14B-ChatEpoch: Qwen-14B-Chat, Alibaba | open | Shots: 0 | 59.5% | not dated | Yi: Open Foundation Models by 01.AI |
| 39 | meta-llama/Llama-2-70b-chatEpoch: Llama-2-70b-chat, Meta AI | open | Shots: 4 | 58.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 40 | gpt-3.5-turbo-0613OpenAI | closed | Shots: 4 | 57.8% | not dated | Baichuan 2: Open Large-scale Language Models |
| 41 | bigcode/starcoder2-15bEpoch: starcoder2-15b, Hugging Face,ServiceNow,Nvidia,BigCode | open | Shots: 4 | 57.7% | not dated | Qwen2.5-Coder Technical Report |
| 42 | text-davinci-003OpenAI | not stated | default | 57.1% | not dated | Yi: Open Foundation Models by 01.AI |
| 43 | text-davinci-003OpenAI | not stated | Shots: 5 | 57.1% | not dated | GPT-4 Technical Report |
| 44 | code-davinci-002OpenAI | closed | Shots: 5 | 56.8% | not dated | Stanford HELM |
| 45 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | default | 56.8% | not dated | Yi: Open Foundation Models by 01.AI |
| 46 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 8 | 56.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 47 | PaLM 540BGoogle Research | closed | default | 56.5% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 48 | falcon-180BTechnology Innovation Instituteno page match | open | default | 54.4% | not dated | Yi: Open Foundation Models by 01.AI |
| 49 | LLaMA-65BMeta AIno page match | open | Shots: 8 | 54.4% | not dated | Qwen Technical Report |
| 50 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | Shots: 8 | 54.4% | not dated | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone |
| 51 | tiiuae/falcon-11BEpoch: falcon-11b, Technology Innovation Institute | open | Shots: 5 | 53.8% | not dated | Falcon2-11B Technical Report |
| 52 | Baichuan-2-13B-BaseBaichuanno page match | open | Shots: 8 | 52.8% | not dated | Nemotron-4 15B Technical Report |
| 53 | Baichuan-2-13B-BaseBaichuanno page match | open | Shots: 4 | 52.8% | not dated | Baichuan 2: Open Large-scale Language Models |
| 54 | Baichuan-2-13B-BaseBaichuanno page match | open | Shots: 8 | 52.8% | not dated | Qwen Technical Report |
| 55 | internlm/internlm-20bEpoch: internlm-20b | open | Shots: 8 | 52.6% | not dated | Qwen Technical Report |
| 56 | Qwen-7BAlibabano page match | open | default | 51.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 57 | Qwen-7BAlibabano page match | open | Shots: 8 | 51.7% | not dated | Qwen Technical Report |
| 58 | LLaMA-65BMeta AIno page match | open | Shots: 8 | 50.9% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 59 | LLaMA-65BMeta AIno page match | open | default | 50.9% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 60 | LLaMA-65BMeta AIno page match | open | Shots: 0 | 50.9% | not dated | Textbooks Are All You Need II: phi-1.5 technical report |
| 61 | text-davinci-003OpenAI | not stated | Shots: 5 | 50.6% | not dated | Stanford HELM |
| 62 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | Shots: 8 | 50.0% | not dated | Mixtral of Experts |
| 63 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 5 | 48.4% | not dated | Stanford HELM |
| 64 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | default | 47.5% | not dated | Yi: Open Foundation Models by 01.AI |
| 65 | meta-llama/Llama-2-70b-chatEpoch: Llama-2-70b-chat, Meta AI | open | Shots: 0 | 47.1% | not dated | Yi: Open Foundation Models by 01.AI |
| 66 | gpt-3.5-turbo-0613OpenAI | closed | Shots: 5 | 46.9% | not dated | Stanford HELM |
| 67 | LLaMA-65BMeta AIno page match | open | Shots: 5 | 46.6% | not dated | Stanford HELM |
| 68 | gemma-7bGoogle DeepMindno page match | open | default | 46.4% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 69 | gemma-7bGoogle DeepMindno page match | open | Shots: 8 | 46.4% | not dated | Nemotron-4 15B Technical Report |
| 70 | Nemotron-4 15BNvidia | closed | Shots: 8 | 46.0% | not dated | Nemotron-4 15B Technical Report |
| 71 | baichuan-inc/Baichuan2-13B-ChatEpoch: Baichuan2-13B-Chat | open | Shots: 0 | 45.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 72 | 01-ai/Yi-6B-ChatEpoch: Yi-6B-Chat, 01.AI | open | Shots: 4 | 44.9% | not dated | Yi: Open Foundation Models by 01.AI |
| 73 | LLaMA-33BMeta AIno page match | open | Shots: 8 | 44.1% | not dated | Mixtral of Experts |
| 74 | internlm/internlm-20bEpoch: internlm-20b | open | Shots: 4 | 43.4% | not dated | Yi: Open Foundation Models by 01.AI |
| 75 | LLaMA-33BMeta AIno page match | open | Shots: 8 | 42.3% | not dated | Qwen Technical Report |
| 76 | Llama-2-34bMeta AI | closed | Shots: 8 | 42.2% | not dated | Nemotron-4 15B Technical Report |
| 77 | Llama-2-34bMeta AI | closed | default | 42.2% | not dated | Yi: Open Foundation Models by 01.AI |
| 78 | Llama-2-34bMeta AI | closed | Shots: 8 | 42.2% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 79 | Llama-2-34bMeta AI | closed | Shots: 8 | 42.2% | not dated | Qwen Technical Report |
| 80 | text-davinci-002OpenAI | closed | Shots: 5 | 41.5% | not dated | Stanford HELM |
| 81 | INTELLECT-1-InstructPrime Intellect,Hugging Face,Arcee AI | not stated | Shots: 5 | 38.6% | not dated | INTELLECT-1 Technical Report |
| 82 | 01-ai/Yi-6B-ChatEpoch: Yi-6B-Chat, 01.AI | open | Shots: 0 | 38.4% | not dated | Yi: Open Foundation Models by 01.AI |
| 83 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | Shots: 5 | 38.1% | not dated | Stanford HELM |
| 84 | Qwen/CodeQwen1.5-7BEpoch: CodeQwen1.5-7B | open | Shots: 4 | 37.7% | not dated | Qwen2.5-Coder Technical Report |
| 85 | meta-llama/Llama-2-13b-chatEpoch: Llama-2-13b-chat, Meta AI | open | Shots: 0 | 36.9% | not dated | Yi: Open Foundation Models by 01.AI |
| 86 | LLaMA-33BMeta AIno page match | open | Shots: 8 | 35.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 87 | LLaMA-33BMeta AIno page match | open | default | 35.6% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 88 | deepseek-ai/deepseek-coder-33b-baseEpoch: deepseek-coder-33b-base, DeepSeek,Peking University | open | Shots: 4 | 35.4% | not dated | Qwen2.5-Coder Technical Report |
| 89 | mistralai/Mistral-7B-Instruct-v0.2Epoch: Mistral-7B-Instruct-v0.2, Mistral AI | open | default | 35.4% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 90 | mistralai/Mistral-7B-v0.1Epoch: Mistral-7B-v0.1, Mistral AI | open | Shots: 8 | 35.4% | not dated | Nemotron-4 15B Technical Report |
| 91 | Qwen/Qwen2.5-Coder-0.5BEpoch: Qwen2.5-Coder-0.5B | open | Shots: 4 | 34.5% | not dated | Qwen2.5-Coder Technical Report |
| 92 | mpt-30b-instructMosaicMLno page match | open | Shots: 5 | 34.4% | not dated | Stanford HELM |
| 93 | Llama-2-13bMeta AIno page match | open | Shots: 8 | 34.3% | not dated | Mixtral of Experts |
| 94 | tiiuae/falcon-40b-instructEpoch: falcon-40b-instruct, Technology Innovation Institute | open | Shots: 5 | 33.8% | not dated | Stanford HELM |
| 95 | PaLM 62B | not stated | default | 33.0% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 96 | bigcode/starcoder2-7bEpoch: starcoder2-7b, Hugging Face,ServiceNow,Nvidia,BigCode | open | Shots: 4 | 32.7% | not dated | Qwen2.5-Coder Technical Report |
| 97 | Yi-6B01.AIno page match | open | default | 32.5% | not dated | Yi: Open Foundation Models by 01.AI |
| 98 | zai-org/chatglm2-6bEpoch: chatglm2-6b | open | Shots: 8 | 32.4% | not dated | Qwen Technical Report |
| 99 | LLaMA-33BMeta AIno page match | open | Shots: 5 | 32.0% | not dated | Stanford HELM |
| 100 | internlm/internlm-7bEpoch: internlm-7b | open | Shots: 8 | 31.2% | not dated | Qwen Technical Report |
| 101 | Llama-2-13bMeta AIno page match | open | Shots: 8 | 29.6% | not dated | Qwen Technical Report |
| 102 | zai-org/chatglm2-6bEpoch: chatglm2-6b | open | Shots: 4 | 28.9% | not dated | Baichuan 2: Open Large-scale Language Models |
| 103 | Llama-2-13bMeta AIno page match | open | Shots: 4 | 28.9% | not dated | Baichuan 2: Open Large-scale Language Models |
| 104 | zai-org/chatglm2-6bEpoch: chatglm2-6b | open | Shots: 8 | 28.8% | not dated | Qwen Technical Report |
| 105 | Llama-2-13bMeta AIno page match | open | default | 28.7% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 106 | Llama-2-13bMeta AIno page match | open | Shots: 8 | 28.7% | not dated | Nemotron-4 15B Technical Report |
| 107 | Llama-2-13bMeta AIno page match | open | Shots: 8 | 28.7% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 108 | vicuna-13b-v1.1 | not stated | Shots: 4 | 28.1% | not dated | Baichuan 2: Open Large-scale Language Models |
| 109 | baichuan-inc/Baichuan-13B-BaseEpoch: Baichuan-13B-Base, Baichuan | open | Shots: 4 | 26.8% | not dated | Baichuan 2: Open Large-scale Language Models |
| 110 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 5 | 25.0% | not dated | Stanford HELM |
| 111 | Baichuan-2-7B-BaseBaichuanno page match | open | Shots: 8 | 24.6% | not dated | Qwen Technical Report |
| 112 | Baichuan-2-7B-BaseBaichuanno page match | open | Shots: 4 | 24.5% | not dated | Baichuan 2: Open Large-scale Language Models |
| 113 | Baichuan-2-7B-BaseBaichuanno page match | open | default | 24.5% | not dated | Yi: Open Foundation Models by 01.AI |
| 114 | Llama-2-13bMeta AIno page match | open | Shots: 5 | 24.5% | not dated | Stanford HELM |
| 115 | baichuan-inc/Baichuan2-13B-ChatEpoch: Baichuan2-13B-Chat | open | Shots: 4 | 23.3% | not dated | Yi: Open Foundation Models by 01.AI |
| 116 | vicuna-13b-v1.3Large Model Systems Organization,University of California (UC) Berkeleyno page match | open | Shots: 5 | 22.6% | not dated | Stanford HELM |
| 117 | Baichuan-2-13B-BaseBaichuanno page match | open | default | 22.1% | not dated | Yi: Open Foundation Models by 01.AI |
| 118 | bigcode/starcoder2-3bEpoch: starcoder2-3b, Hugging Face,ServiceNow,Nvidia,BigCode | open | Shots: 4 | 21.6% | not dated | Qwen2.5-Coder Technical Report |
| 119 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 5 | 21.5% | not dated | Falcon2-11B Technical Report |
| 120 | deepseek-ai/deepseek-coder-6.7b-baseEpoch: deepseek-coder-6.7b-base, DeepSeek,Peking University | open | Shots: 4 | 21.3% | not dated | Qwen2.5-Coder Technical Report |
| 121 | Qwen/Qwen-1_8BEpoch: Qwen-1_8B | open | Shots: 8 | 21.2% | not dated | Qwen Technical Report |
| 122 | LLaMA-13BMeta AIno page match | open | Shots: 4 | 20.5% | not dated | Baichuan 2: Open Large-scale Language Models |
| 123 | LLaMA-13BMeta AIno page match | open | Shots: 8 | 20.3% | not dated | Qwen Technical Report |
| 124 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 8 | 19.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 125 | falcon-40bTechnology Innovation Instituteno page match | open | Shots: 8 | 19.6% | not dated | Qwen Technical Report |
| 126 | LLaMA-13BMeta AIno page match | open | Shots: 8 | 17.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 127 | LLaMA-13BMeta AIno page match | open | default | 17.8% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 128 | gemma-2bGoogle DeepMindno page match | open | default | 17.7% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 129 | Llama-2-7bMeta AIno page match | open | default | 16.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 130 | Llama-2-7bMeta AIno page match | open | Shots: 8 | 16.7% | not dated | Qwen Technical Report |
| 131 | mpt-30bMosaicMLno page match | open | Shots: 5 | 16.4% | not dated | Stanford HELM |
| 132 | Llama-2-7bMeta AIno page match | open | Shots: 4 | 16.2% | not dated | Baichuan 2: Open Large-scale Language Models |
| 133 | Llama-2-7bMeta AIno page match | open | Shots: 8 | 16.0% | not dated | Mixtral of Experts |
| 134 | internlm/internlm-chat-20bEpoch: internlm-chat-20b | open | Shots: 0 | 15.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 135 | LLaMA-13BMeta AIno page match | open | Shots: 5 | 15.4% | not dated | Stanford HELM |
| 136 | mpt-30bMosaicMLno page match | open | Shots: 8 | 15.2% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 137 | mpt-30bMosaicMLno page match | open | Shots: 8 | 15.2% | not dated | Qwen Technical Report |
| 138 | Llama-2-7bMeta AIno page match | open | default | 14.6% | not dated | Gemma: Open Models Based on Gemini Research and Technology |
| 139 | Llama-2-7bMeta AIno page match | open | Shots: 8 | 14.6% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 140 | Llama-2-7bMeta AIno page match | open | Shots: 0 | 14.6% | not dated | Textbooks Are All You Need II: phi-1.5 technical report |
| 141 | meta-llama/Llama-2-70b-hfEpoch: Llama-2-70b-hf, Meta AI | open | Shots: 5 | 13.3% | not dated | Stanford HELM |
| 142 | LLaMA-7BMeta AIno page match | open | Shots: 8 | 11.0% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 143 | LLaMA-7BMeta AIno page match | open | default | 11.0% | not dated | LLaMA: Open and Efficient Foundation Language Models |
| 144 | LLaMA-7BMeta AIno page match | open | Shots: 8 | 11.0% | not dated | Qwen Technical Report |
| 145 | LLaMA-7BMeta AIno page match | open | Shots: 0 | 11.0% | not dated | Textbooks Are All You Need II: phi-1.5 technical report |
| 146 | LLaMA-7BMeta AIno page match | open | Shots: 4 | 9.8% | not dated | Baichuan 2: Open Large-scale Language Models |
| 147 | bigscience/bloomEpoch: bloom, Hugging Face,BigScience | open | Shots: 5 | 9.5% | not dated | Stanford HELM |
| 148 | baichuan-inc/Baichuan-7BEpoch: Baichuan-7B, Baichuan | open | Shots: 4 | 9.2% | not dated | Baichuan 2: Open Large-scale Language Models |
| 149 | mpt-7bMosaicMLno page match | open | Shots: 8 | 9.1% | not dated | Qwen Technical Report |
| 150 | davinciOpenAI | closed | Shots: 5 | 9.0% | not dated | Stanford HELM |
| 151 | mpt-7bMosaicMLno page match | open | Shots: 4 | 8.6% | not dated | Baichuan 2: Open Large-scale Language Models |
| 152 | LLaMA-7BMeta AIno page match | open | Shots: 5 | 8.0% | not dated | Stanford HELM |
| 153 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 8 | 6.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 154 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 8 | 6.8% | not dated | Qwen Technical Report |
| 155 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 0 | 6.8% | not dated | Textbooks Are All You Need II: phi-1.5 technical report |
| 156 | mpt-7bMosaicMLno page match | open | Shots: 8 | 6.8% | not dated | Llama 2: Open Foundation and Fine-Tuned Chat Models |
| 157 | mpt-7bMosaicMLno page match | open | Shots: 0 | 6.8% | not dated | Textbooks Are All You Need II: phi-1.5 technical report |
| 158 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 4 | 5.5% | not dated | Baichuan 2: Open Large-scale Language Models |
| 159 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 5 | 4.6% | not dated | Falcon2-11B Technical Report |
| 160 | deepseek-ai/deepseek-coder-1.3b-baseEpoch: deepseek-coder-1.3b-base, DeepSeek,Peking University | open | Shots: 4 | 4.4% | not dated | Qwen2.5-Coder Technical Report |
| 161 | falcon-7bTechnology Innovation Instituteno page match | open | Shots: 5 | 4.0% | not dated | Stanford HELM |
| 162 | opt-175bMeta AIno page match | open | Shots: 5 | 4.0% | not dated | Stanford HELM |
| 163 | meta-llama/Llama-2-13b-chatEpoch: Llama-2-13b-chat, Meta AI | open | Shots: 4 | 2.7% | not dated | Yi: Open Foundation Models by 01.AI |
| 164 | facebook/opt-66bEpoch: opt-66b, Meta AI | open | Shots: 5 | 1.8% | not dated | Stanford HELM |
| 165 | curieOpenAI | not stated | Shots: 5 | 1.6% | not dated | Stanford HELM |
| 166 | babbageOpenAI | not stated | Shots: 5 | 0.7% | not dated | Stanford HELM |
| 167 | adaOpenAI | not stated | Shots: 5 | 0.6% | not dated | Stanford HELM |
| 168 | text-curie-001OpenAI | closed | Shots: 5 | 0.6% | not dated | Stanford HELM |
| 169 | text-ada-001OpenAI | not stated | Shots: 5 | 0.4% | not dated | Stanford HELM |
| 170 | text-babbage-001OpenAI | closed | Shots: 5 | 0.0% | not dated | Stanford HELM |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method