shimmy
by Michael-A-KuykendallIn its repository's words: “Pure-Rust WebGPU inference engine - OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.”
Facts from GitHub
- Licence
- Apache-2.0 (Apache License 2.0), permissive, our reading
- Activity
- Last push 12 Nov 2026: pushed to in the last 90 days
- Latest release
- not fetched yet
- Stars
- 6.0k
- Main language
- Rust
- On GitHub since
- 28 Aug 2025
- Topics
- api-server, command-line-tool, developer-tools, gguf, huggingface, huggingface-models, huggingface-transformers, inference-server, llama, llamacpp, llm-inference, local-ai, machine-learning, ollama-api, openai-compatible, rust, rust-crate, transformers, webgpu, webgpu-shaders
What it says it supports
- Loads
- GGUF
Each item is on our list for this tool and named in its README. A mention is not a test: check the project's documentation for your setup.
Reviewed models it can load
Matched by file format only, from our snapshot of the Hub: GGUF where we measured GGUF files, other formats where a conversion exists, safetensors for the original weights. Support for a new architecture can lag behind its release.
GGUF 386
openai-community/gpt2, Qwen/Qwen3-0.6B, Qwen/Qwen3-14B, deepseek-ai/DeepSeek-V3.2, Qwen/Qwen3.8-27B, Qwen/Qwen3.5-9B, Qwen/Qwen2.5-7B-Instruct, zai-org/GLM-5.3-Flash, Qwen/Qwen3-VL-8B-Instruct, google/gemma-4-31B-it, Qwen/Qwen3-8B, baidu/Unlimited-OCR, and 374 more, most downloaded first
Other tools in Run locally
Tool data from the GitHub API, fetched 19 Nov 2026, 15:35 UTC. Listing is not endorsement and nobody can pay to be listed. Something wrong here? Write to hello@weightsnotwalls.com.