tract

Examples

examples/ has runnable demos covering the workloads tract targets today:

Resources

Technical documentation lives under doc/ (start at doc/intro.md); the doc/cli-recipe.md page collects practical CLI recipes. The Sonos engineering blog has a long-form post on tract internals.

Python

tract is also available as the tract package on PyPI, built on top of the same Rust core:

pip install tract

The API mirrors the Rust pipeline: load a model, set input facts, optimise, then run. Documentation: sonos.github.io/tract. Source lives in api/py/.

Runtimes

All runtimes share the TypedModel IR and the same loaders, so a model optimised on one platform can be moved to another.

Benchmarks

Every night, main is benchmarked on a fleet of devices — Cortex-A7 to Cortex-A55 boards, RISC-V boards, x86 and Apple desktops, NVIDIA and Apple GPUs, and WebAssembly — over wake-word, speech recognition, speech enhancement, vision and LLM models. The bench page tracks every model on every device. A few of the trends:

Model	Target	Speed	Trend
GTCRN speech enhancement	Cortex-A55	12.1× since Aug 2026
MobileNet v2, multi-thread	Cortex-A53	3.2× since Aug 2026
Parakeet TDT 0.6B preprocessor	Intel i9-11900KB	14.7× since Jun 2026
Llama 3.2 3B text generation	Apple M1 Max, Metal	1.8× since Apr 2025

Speed is relative to the first week of each series, dashed line at 1×. doc/bench/sparklines.py regenerates the charts.

Streaming and pulsification

tract has first-class support for pulsified inference: a network that operates on full sequences during training is translated into one that processes a fixed-size pulse along its streaming axis at each step. This lets the same model serve both batch evaluation and low-latency real-time inference (wake-word, streaming ASR, ...).

The translate-time logic lives in tract-pulse; runtime ships only the small tract-pulse-opl crate. See doc/overview.md § Streaming and pulsification for the engineering view, and examples/nemo-nemotron-streaming-asr for a working demo.

Formats and tract-OPL

PyTorch models can be exported directly to NNEF using torch-to-nnef (source), an open-source PyTorch-to-NNEF converter maintained alongside tract — useful when you want to skip the detour through ONNX.

tract-OPL is an NNEF-compatible intermediate representation that extends NNEF with the operators needed to express a full tract-core model. The recommended deployment workflow is:

1. Once, at build time: convert from ONNX / TF / TFLite to NNEF using the tract CLI:

   tract model.onnx dump --nnef model.nnef.tgz
   

2. At runtime: ship only tract-core + tract-nnef, plus tract-onnx-opl if the model uses ONNX-only operators, and tract-pulse-opl if it is pulsified.

This keeps the runtime footprint small (no protobuf, no training-framework loaders). See doc/intro.md for the full design rationale.

tract-OPL stability

NNEF parts are tied to the NNEF specification and very stable. tract-OPL extensions are a bit more in flux, but we observe the rule:

A model serialised with tract 0.x.y should work with tract 0.x.z where z >= y.

Models embed a tract_nnef_ser_version property identifying the generating tract version; tract itself does not enforce a version check, so it is up to the application to do so if needed. See CHANGELOG.md for the running list of notable serialisation-format changes.

TensorFlow 1 (legacy)

tract still loads TF1 frozen graphs and supports the operator set needed for the classical CV and wake-word models that originally drove its design (Inception v3, Snips wake words, ...). TensorFlow 2 is not directly supported — convert to ONNX first.

License

Files in tensorflow/protos are copied from the TensorFlow project and files in onnx/protos from the ONNX project; neither is covered by the licence statement below.

Apache 2.0 / MIT

All original work is licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any Contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.