Skip to content
Weights Not Walls
Menu

exllamav2

by turboderp-org

In its repository's words: “A fast inference library for running LLMs locally on modern consumer-class GPUs”

Repository on GitHub

Facts from GitHub

Licence
MIT (MIT License), permissive, our reading
Activity
Last push 4 Mar 2026: no push for 90 to 365 days
Latest release
v0.3.213 Jul 2025
Stars
4.7k
Main language
Python
On GitHub since
30 Aug 2023

What it says it supports

Loads
EXL2, GPTQ
Runs on
Linux, Windows
Hardware
NVIDIA

Each item is on our list for this tool and named in its README. A mention is not a test: check the project's documentation for your setup.

Reviewed models it can load

Matched by file format only, from our snapshot of the Hub: GGUF where we measured GGUF files, other formats where a conversion exists, safetensors for the original weights. Support for a new architecture can lag behind its release.

EXL2 23

openai-community/gpt2, Qwen/Qwen2.5-Coder-14B-Instruct, Qwen/Qwen2-VL-2B-Instruct, Qwen/Qwen2.5-32B-Instruct, Qwen/Qwen2-VL-7B-Instruct, Qwen/Qwen2.5-Coder-32B-Instruct, meta-llama/Llama-3.3-70B-Instruct, Qwen/Qwen2.5-14B-Instruct-1M, Qwen/Qwen2.5-72B-Instruct, ibm-granite/granite-3.3-8b-instruct, deepseek-ai/DeepSeek-R1-Distill-Qwen-14B, mistralai/Mistral-Small-24B-Instruct-2501, and 11 more, most downloaded first

GPTQ 142

openai-community/gpt2, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, Qwen/Qwen3.8-27B, Qwen/Qwen2.5-7B-Instruct, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, Qwen/Qwen2.5-VL-7B-Instruct, google/gemma-4-26B-A4B-it, Qwen/Qwen2.5-1.5B-Instruct, google/gemma-4-E4B-it, and 130 more, most downloaded first

Other tools in Run locally

All of Run locally

Tool data from the GitHub API, fetched 19 Nov 2026, 15:36 UTC. Listing is not endorsement and nobody can pay to be listed. Something wrong here? Write to hello@weightsnotwalls.com.