vllm
by vllm-projectIn its repository's words: “A high-throughput and memory-efficient inference and serving engine for LLMs”
Facts from GitHub
- Licence
- Apache-2.0 (Apache License 2.0), permissive, our reading
- Activity
- Last push 19 Nov 2026: pushed to in the last 90 days
- Latest release
- not fetched yet
- Stars
- 97k
- Main language
- Python
- On GitHub since
- 9 Feb 2023
- Topics
- amd, blackwell, cuda, deepseek, deepseek-v3, gpt, gpt-oss, inference, kimi, llama, llm, llm-serving, model-serving, moe, openai, pytorch, qwen, qwen3, tpu, transformer
What it says it supports
- Loads
- safetensors, AWQ, GPTQ, FP8, NVFP4, GGUF, bitsandbytes
- Hardware
- NVIDIA, AMD, Intel, CPU only
Each item is on our list for this tool and named in its README or in docs/features/quantization/README.md, docs/getting_started/installation/README.md. A mention is not a test: check the project's documentation for your setup.
Reviewed models it can load
Matched by file format only, from our snapshot of the Hub: GGUF where we measured GGUF files, other formats where a conversion exists, safetensors for the original weights. Support for a new architecture can lag behind its release.
safetensors 428
openai-community/gpt2, microsoft/Florence-2-base, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, deepseek-ai/DeepSeek-V3.2, Qwen/Qwen3.8-27B, Qwen/Qwen3.5-9B, Qwen/Qwen2.5-7B-Instruct, zai-org/GLM-5.3-Flash, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, and 416 more, most downloaded first
AWQ 213
Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, deepseek-ai/DeepSeek-V3.2, Qwen/Qwen3.8-27B, Qwen/Qwen3.5-9B, Qwen/Qwen2.5-7B-Instruct, zai-org/GLM-5.3-Flash, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, baidu/Unlimited-OCR, Qwen/Qwen3.5-2B, and 201 more, most downloaded first
GPTQ 142
openai-community/gpt2, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, Qwen/Qwen3.8-27B, Qwen/Qwen2.5-7B-Instruct, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, Qwen/Qwen2.5-VL-7B-Instruct, google/gemma-4-26B-A4B-it, Qwen/Qwen2.5-1.5B-Instruct, google/gemma-4-E4B-it, and 130 more, most downloaded first
FP8 235
openai-community/gpt2, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, Qwen/Qwen3.8-27B, Qwen/Qwen3.5-9B, Qwen/Qwen2.5-7B-Instruct, zai-org/GLM-5.3-Flash, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, Qwen/Qwen3.5-2B, Qwen/Qwen2.5-VL-7B-Instruct, and 223 more, most downloaded first
NVFP4 164
Qwen/Qwen3-14B, deepseek-ai/DeepSeek-V3.2, Qwen/Qwen3.8-27B, Qwen/Qwen3.5-9B, zai-org/GLM-5.3-Flash, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, baidu/Unlimited-OCR, Qwen/Qwen3.5-2B, Qwen/Qwen2.5-VL-7B-Instruct, google/gemma-4-26B-A4B-it, and 152 more, most downloaded first
GGUF 386
openai-community/gpt2, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, deepseek-ai/DeepSeek-V3.2, Qwen/Qwen3.8-27B, Qwen/Qwen3.5-9B, Qwen/Qwen2.5-7B-Instruct, zai-org/GLM-5.3-Flash, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, baidu/Unlimited-OCR, and 374 more, most downloaded first
bitsandbytes 161
openai-community/gpt2, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, Qwen/Qwen3.8-27B, Qwen/Qwen2.5-7B-Instruct, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, openai/gpt-oss-20b, Qwen/Qwen2.5-VL-7B-Instruct, Qwen/Qwen2.5-1.5B-Instruct, google/gemma-4-E4B-it, and 149 more, most downloaded first
Other tools in Serve
- LocalAI mudler, 50k stars
- sglang sgl-project, 39k stars
- omlx jundot, 24k stars
- TensorRT-LLM NVIDIA, 15k stars
- OpenLLM bentoml, 13k stars
- text-generation-inference huggingface, 11k stars
Tool data from the GitHub API, fetched 19 Nov 2026, 15:36 UTC. Listing is not endorsement and nobody can pay to be listed. Something wrong here? Write to hello@weightsnotwalls.com.