The state of ML in Rust, as of 9 Nov 2026

mistralrs: Local LLM inference behind a Rust SDK

Default features build in pure Rust, loads GGUF and multimodal models, and exposes tools, structured output and an MCP client behind a single SDK.

tantivy: Full-text search library with BM25

BM25, facets, fast fields and mmap in a library other Rust search engines build on; you write the server layer.

qdrant: Vector search server at production scale

Rust server with sharding, replication, quantization and sparse plus multi-vector search, reached over REST and gRPC.

rmcp: Build MCP servers and clients

Official MCP SDK with tools, resources, transports and OAuth; other Rust MCP crates sit on top of it.

liter-llm: Call hosted model APIs from Rust

A single async client over many providers with tools, streaming and structured output, so provider churn stays out of your app.

earshot: Streaming voice activity detection

Pure-Rust streaming VAD over short frames, returns a voice score, builds without ONNX Runtime and can run no_std.

rustfft: FFT of any size in pure Rust

SIMD-accelerated transforms of any size including primes; the FFT underneath ndarray and other crates.

faer: Dense linear algebra and decompositions

Dense linear algebra with decompositions and a high-level API, written in pure Rust so no LAPACK link is needed.

ndarray: N-dimensional arrays for numeric code

The array container most Rust ML crates share; slicing, views, Zip and optional BLAS keep numeric code portable.

safetensors: Read and write model weights

Zero-copy reads and safe writes for model weights; the interchange format Rust trainers and loaders already speak.

hf-hub: Fetch models and datasets from the Hub

Async and blocking client for Hub models, datasets and files, native and wasm; keeps download and cache logic out of your code.

tokenizers: Exact tokenization for model families

Reference BPE, WordPiece and Unigram pipeline with tokenizer.json load and save; matches the main model families.

oxionnx: Pure-Rust ONNX inference engine

No C or C++ in the build, and real detection models run. Still needs a tokenizer and sampler bolted on, and browser WebGPU is unwired.

treeboost: Gradient-boosted trees for tabular data

Boosted trees and forests with an AutoML mode and SIMD plus GPU backends. No changelog, unstable API and a small team behind it.

oxidizr: Config-driven language-model training

Runs the training loop with checkpoints, metrics and resume on a local machine. No FSDP, ZeRO, tensor or pipeline parallelism.

vidr: GPU video decode, encode and upscaling

Decode, encode, muxing and upscaling in one workspace with frames kept on the GPU. Early release, NVIDIA-tested, no VA-API, H.264 Baseline only.

franken_ocr: Pure-Rust CPU OCR for documents

Runs document and formula OCR models with Markdown and box output and no FFI. Lone maintainer, CPU only, not a general model runtime.

model2vec-rs: Static text embeddings, WASM included

Static embeddings with int8 quantization and a WASM build, no ONNX Runtime. No training or distillation, and no GPU path.

Quantized LLM inference plus an OpenAI-compatible server

Teams can serve a GGUF model without PyTorch; checkpoint conversion and quantization still use Python first.

Use blazr, llama-gguf, candle-vllm, furnace.

ONNX inference with no C or C++ runtime

Exported models run in Rust on CPU, Metal, CUDA or WASM; operator coverage and tokenizer or sampler wiring are the cost.

Use tract, oxionnx, lele.

Tokenization that matches the main model families

Exact ids without a Python round trip; validate against reference output for the families you depend on.

Use tokenizers, tiktoken-rs, kitoken.

Full-text and vector search over your own data

Both halves of hybrid search exist in Rust; reranking still runs through wrapped ONNX Runtime.

Use tantivy, qdrant, seekstorm.

Streaming voice activity detection without a native runtime

Gate microphones in pure Rust; learned VAD through ONNX Runtime remains an option when accuracy demands it.

Use earshot, sonora.

Numerical core: arrays, FFT and dense linear algebra

Most SciPy work on a single machine translates; plotting and reference validation stay in Python.

Use ndarray, rustfft, faer.

Model and dataset IO for weights and columnar data

Read and write the formats trainers and loaders use; legacy PyTorch checkpoints still need conversion tooling.

Use safetensors, gguf, arrow-rs.

XGBoost and LightGBM parity for tabular models

Teams with existing boosted-tree pipelines cannot port them and compare like for like in Rust.

What teams do today: Train with treeboost or perpetual and export to XGBoost or ONNX, or keep training in Python.

Rust-native inference on phones

Mobile teams fall back to llama.cpp, whisper.cpp or ONNX Runtime bindings, so the build is not Rust end to end.

What teams do today: xybrid ships Android and iOS presets but wraps native engines; needle-rs covers tool-calling models only and leaves NPU work to Cactus.

Maintained multi-object tracking across video frames

Video analytics teams cannot keep detection-to-track state on a maintained Rust crate; the only tracker row is unmaintained.

What teams do today: similari has Kalman, SORT, Batch SORT and VisualSORT but is unmaintained; teams write IoU and Hungarian assignment on ndarray or use Python ByteTrack.

A maintained reinforcement-learning trainer

RL teams get environments but no maintained learner; the older Rust RL crates are unmaintained.

What teams do today: Use bevy_rl for environments and write the learner on burn or candle, or keep training in Python.

VA-API and H.264 High profile in Rust video

Linux teams without NVIDIA cannot decode or encode common camera and screen streams through a Rust path.

What teams do today: vidr covers NVDEC, NVENC and Vulkan Video plus Baseline H.264 and AV1 on CPU; otherwise transcode with FFmpeg or a Python pipeline.