VideoMME
ReportedNo description yet: we only describe an eval set when we can do it correctly.
- Kind
- Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
- Link
- https://video-mme.github.io/home_page.html#leaderboard
- Score shown
- Epoch's column "Overall (no subtitles)", as a percentage, higher is better. The source gives no error bars.
- Results
- 50 in all: 10 for open-weight models, 9 for closed ones, 31 where Epoch does not say. 8 linked to a model page here.
Best open-weight result: Qwen2.5-VL-72B-Instruct, 73.5%, number 4 of 50 results.
| # | Model | Weights | Setting | Score | Date | Source |
|---|---|---|---|---|---|---|
| 1 | video-SALMONN-2plusByteDance | not stated | default | 79.7% | not dated | Video-MME Leaderboard, via Epoch AI |
| 2 | gemini-1.5-pro-001Google DeepMind | closed | default | 75.0% | not dated | video-mme.github.io |
| 3 | gemini-1.5-pro-001-feb24Google DeepMind | closed | default | 75.0% | not dated | Video-MME Leaderboard |
| 4 | Qwen2.5-VL-72B-InstructAlibabano page match | open | default | 73.5% | not dated | Video-MME Leaderboard |
| 5 | OpenGVLab/InternVL2_5-78BEpoch: InternVL2_5-78B, Shanghai AI Lab,SenseTime,Tsinghua University,Nanjing University,Fudan University,Chinese University of Hong Kong (CUHK),Shanghai Jiao Tong University | open | default | 72.1% | not dated | Video-MME Leaderboard |
| 6 | gpt-4o-2024-08-06OpenAI | closed | default | 71.9% | not dated | Video-MME Leaderboard |
| 7 | gpt-4o-2024-11-20OpenAI | closed | default | 71.9% | not dated | Video-MME Leaderboard, via Epoch AI |
| 8 | Qwen/Qwen2-VL-72B-InstructEpoch: Qwen2-VL-72B-Instruct | open | default | 71.2% | not dated | Video-MME Leaderboard |
| 9 | LLaVA-Video-72B-Qwen2 | not stated | default | 70.6% | not dated | Video-MME Leaderboard |
| 10 | gemini-1.5-flash-001Google DeepMind | closed | default | 70.3% | not dated | Video-MME Leaderboard |
| 11 | LinVT | not stated | default | 70.3% | not dated | Video-MME Leaderboard |
| 12 | Aria | not stated | default | 67.6% | not dated | Video-MME Leaderboard |
| 13 | ViLAMP-llava-qwen | not stated | default | 67.5% | not dated | Video-MME Leaderboard |
| 14 | Oryx-1.5-32B | not stated | default | 67.3% | not dated | Video-MME Leaderboard |
| 15 | LLaVA-OneVision 72B | not stated | default | 66.3% | not dated | Video-MME Leaderboard |
| 16 | VideoLLAMA3-7B | not stated | default | 66.2% | not dated | Video-MME Leaderboard |
| 17 | LLaVA-Video-7B-Qwen2 | not stated | default | 65.9% | not dated | Video-MME Leaderboard |
| 18 | LLaVA-Video-7B-Qwen2-TPO | not stated | default | 65.6% | not dated | Video-MME Leaderboard |
| 19 | VideoChat-Flash-Qwen2-7B_res448 | not stated | default | 65.3% | not dated | Video-MME Leaderboard |
| 20 | gpt-4o-mini-2024-07-18OpenAI | closed | default | 64.8% | not dated | Video-MME Leaderboard |
| 21 | ByteVideoLLM-14B | not stated | default | 64.6% | not dated | Video-MME Leaderboard |
| 22 | NVILA-8BNvidia,Massachusetts Institute of Technology (MIT),University of California (UC) Berkeley,University of California San Diego,University of Washington,Tsinghua Universityno page match | open | default | 64.2% | not dated | Video-MME Leaderboard |
| 23 | LiveCC-7B-Instruct | not stated | default | 64.1% | not dated | Video-MME Leaderboard |
| 24 | openbmb/MiniCPM-o-2_6Epoch: MiniCPM-o-2_6 | open | default | 63.9% | not dated | Video-MME Leaderboard |
| 25 | Qwen/Qwen2-VL-7B-InstructEpoch: Qwen2-VL-7B-Instruct | open | default | 63.9% | not dated | Video-MME Leaderboard |
| 26 | VideoLLAMA2-7B | not stated | default | 62.4% | not dated | Video-MME Leaderboard |
| 27 | OpenGVLab/InternVL2-40BEpoch: InternVL2-40B, Shanghai AI Lab | open | default | 61.2% | not dated | Video-MME Leaderboard |
| 28 | openbmb/MiniCPM-V-2_6Epoch: MiniCPM-V-2_6 | open | default | 60.9% | not dated | Video-MME Leaderboard |
| 29 | claude-3-5-sonnet-20240620Anthropic | closed | default | 60.0% | not dated | Video-MME Leaderboard |
| 30 | claude-3-5-sonnet-20241022Anthropic | closed | default | 60.0% | not dated | Video-MME Leaderboard, via Epoch AI |
| 31 | gpt-4-1106-vision-previewOpenAI | closed | default | 59.9% | not dated | video-mme.github.io |
| 32 | mPLUG-Owl3-7B-241101 | not stated | default | 59.3% | not dated | Video-MME Leaderboard |
| 33 | TimeMarker | not stated | default | 57.3% | not dated | Video-MME Leaderboard |
| 34 | VITA-1.5 | not stated | default | 56.1% | not dated | Video-MME Leaderboard |
| 35 | kangaroo | not stated | default | 56.0% | not dated | Video-MME Leaderboard |
| 36 | VITA | not stated | default | 55.8% | not dated | Video-MME Leaderboard |
| 37 | Video-XL-7B | not stated | default | 55.5% | not dated | Video-MME Leaderboard |
| 38 | Video-CCAM-7B-v1.2 | not stated | default | 53.2% | not dated | Video-MME Leaderboard |
| 39 | long-llava-qwen2-7b | not stated | default | 52.9% | not dated | Video-MME Leaderboard |
| 40 | LongVA-7B | not stated | default | 52.6% | not dated | Video-MME Leaderboard |
| 41 | Qwen-VL-Max | not stated | default | 51.3% | not dated | Video-MME Leaderboard |
| 42 | OpenGVLab/InternVL-Chat-V1-5Epoch: InternVL-Chat-V1-5 | open | default | 50.7% | not dated | Video-MME Leaderboard |
| 43 | SliME-Llama3-8B | not stated | default | 45.3% | not dated | Video-MME Leaderboard |
| 44 | Chat-Uni-Vi-7B-v1.5 + 100k SG-WV | not stated | default | 43.2% | not dated | Video-MME Leaderboard |
| 45 | Qwen/Qwen-VL-ChatEpoch: Qwen-VL-Chat | open | default | 41.1% | not dated | Video-MME Leaderboard |
| 46 | Chat-UniVi-7B-v1.5 | not stated | default | 40.6% | not dated | Video-MME Leaderboard |
| 47 | sharegpt4video-8b | not stated | default | 39.9% | not dated | Video-MME Leaderboard |
| 48 | Video-LLaVA-7B | not stated | default | 39.9% | not dated | Video-MME Leaderboard |
| 49 | video_chat2_mistral | not stated | default | 39.5% | not dated | Video-MME Leaderboard |
| 50 | ST-LLM | not stated | default | 37.9% | not dated | Video-MME Leaderboard |
Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method