Skip to content
Weights Not Walls
Menu

VideoMME

Reported

No description yet: we only describe an eval set when we can do it correctly.

Kind
Published by a third party's leaderboard, collected by Epoch AI. The leaderboard is named and linked.
Link
https://video-mme.github.io/home_page.html#leaderboard
Score shown
Epoch's column "Overall (no subtitles)", as a percentage, higher is better. The source gives no error bars.
Results
50 in all: 10 for open-weight models, 9 for closed ones, 31 where Epoch does not say. 8 linked to a model page here.

Best open-weight result: Qwen2.5-VL-72B-Instruct, 73.5%, number 4 of 50 results.

VideoMME results
#ModelWeightsSettingScoreDateSource
1video-SALMONN-2plusByteDancenot stateddefault79.7%not datedVideo-MME Leaderboard, via Epoch AI
2gemini-1.5-pro-001Google DeepMindcloseddefault75.0%not datedvideo-mme.github.io
3gemini-1.5-pro-001-feb24Google DeepMindcloseddefault75.0%not datedVideo-MME Leaderboard
4Qwen2.5-VL-72B-InstructAlibabano page matchopendefault73.5%not datedVideo-MME Leaderboard
5OpenGVLab/InternVL2_5-78BEpoch: InternVL2_5-78B, Shanghai AI Lab,SenseTime,Tsinghua University,Nanjing University,Fudan University,Chinese University of Hong Kong (CUHK),Shanghai Jiao Tong Universityopendefault72.1%not datedVideo-MME Leaderboard
6gpt-4o-2024-08-06OpenAIcloseddefault71.9%not datedVideo-MME Leaderboard
7gpt-4o-2024-11-20OpenAIcloseddefault71.9%not datedVideo-MME Leaderboard, via Epoch AI
8Qwen/Qwen2-VL-72B-InstructEpoch: Qwen2-VL-72B-Instructopendefault71.2%not datedVideo-MME Leaderboard
9LLaVA-Video-72B-Qwen2not stateddefault70.6%not datedVideo-MME Leaderboard
10gemini-1.5-flash-001Google DeepMindcloseddefault70.3%not datedVideo-MME Leaderboard
11LinVTnot stateddefault70.3%not datedVideo-MME Leaderboard
12Arianot stateddefault67.6%not datedVideo-MME Leaderboard
13ViLAMP-llava-qwennot stateddefault67.5%not datedVideo-MME Leaderboard
14Oryx-1.5-32Bnot stateddefault67.3%not datedVideo-MME Leaderboard
15LLaVA-OneVision 72Bnot stateddefault66.3%not datedVideo-MME Leaderboard
16VideoLLAMA3-7Bnot stateddefault66.2%not datedVideo-MME Leaderboard
17LLaVA-Video-7B-Qwen2not stateddefault65.9%not datedVideo-MME Leaderboard
18LLaVA-Video-7B-Qwen2-TPOnot stateddefault65.6%not datedVideo-MME Leaderboard
19VideoChat-Flash-Qwen2-7B_res448not stateddefault65.3%not datedVideo-MME Leaderboard
20gpt-4o-mini-2024-07-18OpenAIcloseddefault64.8%not datedVideo-MME Leaderboard
21ByteVideoLLM-14Bnot stateddefault64.6%not datedVideo-MME Leaderboard
22NVILA-8BNvidia,Massachusetts Institute of Technology (MIT),University of California (UC) Berkeley,University of California San Diego,University of Washington,Tsinghua Universityno page matchopendefault64.2%not datedVideo-MME Leaderboard
23LiveCC-7B-Instructnot stateddefault64.1%not datedVideo-MME Leaderboard
24openbmb/MiniCPM-o-2_6Epoch: MiniCPM-o-2_6opendefault63.9%not datedVideo-MME Leaderboard
25Qwen/Qwen2-VL-7B-InstructEpoch: Qwen2-VL-7B-Instructopendefault63.9%not datedVideo-MME Leaderboard
26VideoLLAMA2-7Bnot stateddefault62.4%not datedVideo-MME Leaderboard
27OpenGVLab/InternVL2-40BEpoch: InternVL2-40B, Shanghai AI Labopendefault61.2%not datedVideo-MME Leaderboard
28openbmb/MiniCPM-V-2_6Epoch: MiniCPM-V-2_6opendefault60.9%not datedVideo-MME Leaderboard
29claude-3-5-sonnet-20240620Anthropiccloseddefault60.0%not datedVideo-MME Leaderboard
30claude-3-5-sonnet-20241022Anthropiccloseddefault60.0%not datedVideo-MME Leaderboard, via Epoch AI
31gpt-4-1106-vision-previewOpenAIcloseddefault59.9%not datedvideo-mme.github.io
32mPLUG-Owl3-7B-241101not stateddefault59.3%not datedVideo-MME Leaderboard
33TimeMarkernot stateddefault57.3%not datedVideo-MME Leaderboard
34VITA-1.5not stateddefault56.1%not datedVideo-MME Leaderboard
35kangaroonot stateddefault56.0%not datedVideo-MME Leaderboard
36VITAnot stateddefault55.8%not datedVideo-MME Leaderboard
37Video-XL-7Bnot stateddefault55.5%not datedVideo-MME Leaderboard
38Video-CCAM-7B-v1.2not stateddefault53.2%not datedVideo-MME Leaderboard
39long-llava-qwen2-7bnot stateddefault52.9%not datedVideo-MME Leaderboard
40LongVA-7Bnot stateddefault52.6%not datedVideo-MME Leaderboard
41Qwen-VL-Maxnot stateddefault51.3%not datedVideo-MME Leaderboard
42OpenGVLab/InternVL-Chat-V1-5Epoch: InternVL-Chat-V1-5opendefault50.7%not datedVideo-MME Leaderboard
43SliME-Llama3-8Bnot stateddefault45.3%not datedVideo-MME Leaderboard
44Chat-Uni-Vi-7B-v1.5 + 100k SG-WVnot stateddefault43.2%not datedVideo-MME Leaderboard
45Qwen/Qwen-VL-ChatEpoch: Qwen-VL-Chatopendefault41.1%not datedVideo-MME Leaderboard
46Chat-UniVi-7B-v1.5not stateddefault40.6%not datedVideo-MME Leaderboard
47sharegpt4video-8bnot stateddefault39.9%not datedVideo-MME Leaderboard
48Video-LLaVA-7Bnot stateddefault39.9%not datedVideo-MME Leaderboard
49video_chat2_mistralnot stateddefault39.5%not datedVideo-MME Leaderboard
50ST-LLMnot stateddefault37.9%not datedVideo-MME Leaderboard

Open-weight rows are shaded. Dates are when the result was run or evaluated, where the data says. Data from Epoch AI (CC-BY 4.0), fetched 19 Nov 2026: Epoch AI, ‘Capabilities & benchmarking’. Published online at epoch.ai. Retrieved from ‘https://epoch.ai/benchmarks’ [online resource]. Epoch AI. Method