shimmy
App Listed
Lightweight Ollama-compatible inference server with native SafeTensors support. No Python dependencies, cross-platform WebGPU acceleration via Airframe.
- Downloads
- 352
- Dependents
- Not applicable
- Stars
- 6,014
- Forks
- 587
- Contributors
- 1
- Commits
- 225
- Latest release
- 2.6.6 ·
About
Shimmy — Local Inference, OpenAI-Compatible
🔒 Local GPU inference — Rust/WGSL application 🚀
Shimmy 2.6.6
Shimmy is a local inference server and CLI; Airframe 0.4.5 is its engine. This is a coordinated dense-text correctness, dependency-security and documentation release, not a new architecture or performance release. See CHANGELOG.
Default dependency licensing: Shimmy → Airframe → libfse. Shimmy is Apache-2.0; Airframe and airframe-observe are MIT; ShimmyTok is MIT OR Apache-2.0. The libfse 0.1.4 additional Apache-2.0 grant permits commercial use, modification, redistribution and incorporation into products without separate commercial permission. The original crates.io archive and metadata remain unchanged. Preserve component and third-party notices; see stack licensing.
Install and run
Verified native 2.6.6 archives and checksum instructions are available separately from crates.io, including the Windows executable and complete licensing bundles.
Install the versioned crate with a Rust toolchain:
cargo install shimmy --version 2.6.6 --locked
shimmy --version
shimmy --ctx 512 --kv-quant f32 serve --model-path /path/to/model.gguf --bind 127.0.0.1:11435
Read the full README
Use a verified admitted fixture in place of the path. A working Rust toolchain and GPU driver are required. The fast profile does not provide inference. The separately verified native downloads, Homebrew and Scoop installers and versioned GHCR image are now available for 2.6.6. A crates.io upload does not create those channels automatically. Docker latest is unchanged; npm is not included. Verify the exact installed version.
See quickstart for model hashes, listing, requests and network access. Unavailable translations are not linked as current guides.
Correctness and support
The exact fixture matrix covers selected Qwen3-0.6B/Qwen3-1.7B and Llama 3.2 3B Q4_K_M cases, with independent FP32 full logits, maximum absolute 0.1/RMS 0.03 gates and matching measured greedy decisions. These are bounded model/context/mode results, not universal family support.
F32 KV and reference kernels define the validated path. Parsing metadata, tokenizers or quantization is not certification. Gemma2, MoE, MLA, recurrent, state-space and multimodal executors are not enabled. INT4/TurboShimmy and experimental kernels remain excluded and rejected.
Executed full-logit GPU evidence is NVIDIA through WSL Dozen/Vulkan. NOT_RUN for full-logit certification: Metal/M4, other unexecuted native GPU platforms, extended-context Llama, alternate prefill chunks and unlisted combinations. WSL device-destruction and active-generation device-loss investigations remain open. No universal determinism, startup, multi-GPU, memory or speed guarantee is made.
Server contract
Selected OpenAI-compatible endpoints provide model listing, JSON chat/text completions and explicit chat-only SSE. Chat defaults to JSON; text streaming rejects. Usage, stop/finish metadata and typed errors are part of the tested contract. This is not every OpenAI endpoint/SDK feature.
Loopback can run without a key. Non-loopback binds require SHIMMY_API_KEY; browser origins require SHIMMY_ALLOWED_ORIGINS. With a key, routes except GET /health require credentials; origin checks still apply to health. Use TLS and appropriate network access controls remotely. Lazy loading is the default; optional selected-model warm-up shifts cold work before readiness without eliminating it or guaranteeing stability.
Documentation
- Quickstart and configuration
- API, compatibility and integration
- Model scope, certification and quantization
- Context capacity and templates
- Docker scope, Windows scope and troubleshooting
- Security, release process and release gates
- Developer guide, contribution policy and roadmap
Shimmy source is Apache-2.0. Airframe runtime and Observe are MIT; ShimmyTok is dual-licensed. libfse 0.1.4 is covered by the additional Apache-2.0 grant, permitting commercial use without separate permission. Preserve applicable component, dependency and model terms; see stack licensing.
Support
This project is independently maintained. Component licensing terms described above apply. Sponsorship helps fund ongoing development, certification, compatibility work, and releases.
Sponsor on GitHub · Buy Me a Coffee
Any amount is appreciated. Suggested tiers:
- $5/month — Thanks
- $15/month — Supporter
- $50/month — Backer
- Custom — Email for invoicing
License & Philosophy
Shimmy server code: Apache-2.0 — see LICENSE. The default dependency stack permits commercial use under its component terms, including the libfse additional grant. See stack licensing; original published archives remain unchanged.
Promise: This will never become a paid product.
Shimmy is infrastructure: it should be invisible. Reliability through comprehensive validation and property-based testing.
Maintainer: Michael A. Kuykendall · Mission: Making local model inference simple and reliable
Support
This project is a safe space. Trans rights are human rights.
If you or someone you love needs support:
- The Trevor Project — 24/7 for LGBTQ+ young people. Call 1-866-488-7386 or text START to 678-678
- Trans Lifeline — peer support run by and for trans people. US: 877-565-8860
- 988 Suicide & Crisis Lifeline — call or text 988
Compatibility
- Runs on
- GPU support unknown
What shows it
- no dependency or feature matches a backend row
- Native code
- Required
- Binds native code: sys-info
- Python bindings
- No Python binding crate found
- License
- Apache-2.0
Compare with similar projects
Order: strongest tier first, then most downloads, then name, with unmeasured values last. A library counts downloads over 30 days. An app counts binary downloads per release. An app with none counts its crate downloads over 30 days. ML Rust runs this catalogue, so its closest project always gets a row.
| Project | Downloads | Dependents | Stars | Latest release | Runs on (GPU) | Native code | License |
|---|---|---|---|---|---|---|---|
|
Lightweight Ollama-compatible inference server with native SafeTensors support. No Python dependencies, cross-platform WebGPU acceleration via Airframe. |
352 binary downloads per release | Not applicable | 6,014 | 2.6.6 | GPU support unknown | Binds native code: sys-info | Apache-2.0 |
|
orp Listed
Lightweight framework for building ONNX runtime pipelines with ort Also in Run and serve models. |
41,463 downloads over 30 days | 6 | 14 | 1.0.0 | CUDA, ROCm | Binds native code: ort-sys | Apache-2.0 |
|
oxionnx Listed
Pure Rust ONNX inference engine — zero C/C++ dependencies Also in Run and serve models. |
550 downloads over 30 days | 24 | 20 | 0.1.7 | CUDA, WebGPU | No native code found | Apache-2.0 |
|
Efficent platform for inference and serving local LLMs including an OpenAI compatible API server. Also in Build LLM apps and agents. |
138 binary downloads per release | Not applicable | 739 | Not measured | Not measured | MIT | |
|
Inference server for LLMs, speech and embeddings: OpenAI-compatible routes with streaming, plus speech synthesis (TTS) and transcription (ASR). Serves Llama, Mistral, Qwen, Mamba2, MLA+MoE and hybrid models from SafeTensors, AWQ, GPTQ and GGUF. Also in Build LLM apps and agents. |
8 binary downloads per release | Not applicable | 7 | 0.7.1 | CUDA, WebGPU | Binds native code: ring (feature distributed) | Apache-2.0 |
|
furnace New
Fast, unified, Rust-native, Axum-compatible inference endpoints and nervous system for tensor models and LLMs: KServe / Triton and OpenAI APIs from one pure-Rust server Also in Build LLM apps and agents. |
241 downloads over 30 days | 0 | 66 | 0.16.2 | CUDA, WebGPU, Metal | No native code found | MIT |
All projects in Build LLM apps and agentsAll projects in Run and serve models
Why ListedNo archive, New or Featured rule places it, so it is Listed.
What it lacks for Featured
- The check holds no value for Homebrew installs on request over 365 days, Repology repository families, Docker pull growth over 90 days. A gate that lacks its signal does not pass.
What else the check found
- The last activity is on 7 Nov 2026, 2 days ago. The rules expect an update within 365 days. Dormancy rule
Not measured: Homebrew installs on request over 365 days, Repology repository families, Docker pull growth over 90 days.
Exact rules: Listed, Featured app. The check on applied thresholds version 9.
Every rule outcome, as the check records it
expected-interval: app: E = 365 daysdormancy: last activity 2026-11-07, 2 days ago, E = 365: activefeatured: not featured: unknown homebrew_on_request_365d, repology_families, docker_pull_growth_90dlisted: listed: no archive, new or featured rule applies
Measurements and sources
Each signal row shows the raw value, its source and the fetch date. The Sources list gathers every distinct source on this page. Movement needs more history.
Sources6 sources
- Description: the crates.io description
- crates.io database dump, fetched
- GitHub releases, fetched
- GitHub repository, fetched
- crates.io package, fetched
- GitHub contributors, fetched
Adoption352 binary downloads per release4 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Active dependents | Not measured | Yes | ||
| New dependents (365 days) | Not measured | Yes | ||
| Peak active dependents | Not measured | Yes | ||
| Active dependents as a share of the peak | Not measured | Yes | ||
| Binary downloads per release | 352 | GitHub releases | Yes |
MaintenanceLatest human commit: 2 days ago4 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Latest human commit | 2026-11-07 | GitHub repository | Yes | |
| Latest release | 2026-11-07 | GitHub releases | Yes | |
| First release | 2025-09-04 | crates.io database dump | Yes | |
| Latest version | 2.6.6 | crates.io database dump | No | |
| Days the oldest outside issue waits for a reply | 46 | GitHub repository | Yes | |
| Maintainer replies over 365 days | 82 | GitHub repository | Yes | |
| Repository archived | false | GitHub repository | Yes | |
| Repository gone | false | GitHub repository | Yes | |
| Release commit or manifest in the repo | true | crates.io package | No | |
| Published docs | Not measured | Yes | ||
| Latest release fails cargo check | Not measured | Yes | ||
| Build check date | Not measured | Yes | ||
| Build check toolchain | Not measured | Yes |
SecurityNot measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Unpatched vulnerabilities past the grace window | Not measured | Yes | ||
| RustSec unmaintained advisory | Not measured | Yes | ||
| Deprecation notice | Not measured | Yes | ||
| Deprecated requirement | Not measured | Yes |
Distribution34 releases, 259 downloads over 30 days6 not measured
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| Releases | 34 | crates.io database dump | No | |
| Downloads over 30 days | 259 | crates.io database dump | No | |
| Binary downloads over the last 3 releases | 1057 | GitHub releases | Yes | |
| Homebrew installs on request over 365 days | Not measured | Yes | ||
| Repology repository families | Not measured | Yes | ||
| Docker pull growth over 90 days | Not measured | Yes | ||
| Docker pulls at the growth baseline | Not measured | Yes | ||
| Every version a prerelease | false | crates.io database dump | Yes | |
| Every version 0.0.x | false | crates.io database dump | Yes | |
| Every version yanked | Not measured | Yes | ||
| Crate deleted from crates.io | Not measured | Yes |
Repository6,014 stars, 587 forks, 1 contributors, 225 commits. Stars alone never qualify a project.
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| GitHub stars | 6014 | GitHub repository | Yes — one part of the repository route | |
| GitHub forks | 587 | GitHub repository | Yes — one part of the repository route | |
| Contributors with a GitHub account | 1 | GitHub contributors | Yes — one part of the repository route | |
| Commits on the default branch | 225 | GitHub repository | No | |
| Repository's main crate left out of the entry | none | crates.io database dump | Yes |
Facts that never count
| Signal | Value | Source | Fetched | Counts |
|---|---|---|---|---|
| License | Apache-2.0 | crates.io database dump | No | |
| Runs on, from listed crates and their direct dependencies | GPU support unknown | crates.io database dump | No | |
| What shows each backend | no dependency or feature matches a backend row | crates.io database dump | No | |
| Native code, from listed crates and their dependencies | Binds native code: sys-info | crates.io database dump | No | |
| Python bindings, from direct dependencies of listed crates | No Python binding crate found | crates.io database dump | No | |
| Crates that depend on a listed crate directly | Not measured | No |
Corrections and review history
No correction applies to this entry.
Judgments
A judgment enters only through an approved issue and a merged pull request.
No judgment applies to this entry.