lmms-eval
by EvolvingLMMs-LabIn its repository's words: “One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks”
Facts from GitHub
- Licence
- MIT AND Apache-2.0 (MIT License and Apache License 2.0), permissive, our readingGitHub's licence detection could not identify the licence file, so we read the file on 19 Nov 2026: MIT for the pipeline code taken from lm-evaluation-harness, Apache License 2.0 for the task and model code.
- Activity
- Last push 18 Nov 2026: pushed to in the last 90 days
- Latest release
- not fetched yet
- Stars
- 4.5k
- Main language
- Python
- On GitHub since
- 7 Mar 2024
- Topics
- agi, audio-evaluation, benchmark, evaluation, large-language-models, llm-evaluation, multimodal, multimodal-evaluation, video-understanding, vision-language-model, vlm
Other tools in Evaluate
- promptfoo promptfoo, 27k stars
- deepeval confident-ai, 20k stars
- lm-evaluation-harness EleutherAI, 15k stars
- garak NVIDIA, 10k stars
- opencompass open-compass, 7.6k stars
- SWE-bench SWE-bench, 6.2k stars
Tool data from the GitHub API, fetched 19 Nov 2026, 15:34 UTC. Listing is not endorsement and nobody can pay to be listed. Something wrong here? Write to hello@weightsnotwalls.com.