transcribe-cpp

Speaker diarization models (no transcription; verified by DER/JER rather than WER):

Language ID models (no transcription; verified by top-1 decision parity on FLEURS; see docs/langid.md):

Voice activity detection models (no transcription; see docs/vad.md):

Per-variant model cards live under docs/models/.

Model catalog

catalog/ is the source of truth for model metadata, accuracy, and performance. Each release includes a queryable catalog.db and SHA-256 checksum. Rebuild it locally with uv run scripts/catalog/db.py --out catalog.db. The exact accuracy and published speed matrices and standard benchmark recipes required for publication live in catalog/_benchmark_profiles.json. Run uv run scripts/catalog/check.py --publication-profile to enforce it; the ordinary catalog check reports the migration backlog without failing.

Published tables are generated from it rather than hand-written. A model doc delegates a region with a marker, and scripts/catalog/render.py rewrites only what sits between the pair:

<!-- catalog:downloads -->
| Quantization | Download | Size | WER (LibriSpeech test-clean) |
...
<!-- /catalog -->

The Hugging Face card specs under scripts/hf_cards/ hold editorial copy only (summary, tags, validation pin, prose notes). scripts/hf_cards/generate.py reads the spec and the catalog record together, so repos, licence, languages, capabilities, the quant table, and per-rig speedups are never written into a YAML by hand.

uv run scripts/catalog/format.py --check  # canonical record layout
uv run scripts/catalog/check.py           # schema, integrity, pairing
uv run scripts/catalog/render.py          # rewrite the marked doc regions
uv run scripts/catalog/render.py --check

Build

cmake -B build
cmake --build build

Metal is enabled automatically on Apple Silicon. For Vulkan (Linux/Windows):

# Ubuntu/Debian
sudo apt install build-essential cmake libvulkan-dev glslc libopenblas-dev

# Fedora
sudo dnf install vulkan-headers openblas-devel glslc spirv-headers-devel vulkan-loader-devel

cmake -B build -DTRANSCRIBE_VULKAN=ON
cmake --build build

On Windows, see the complete build guide for Vulkan SDK setup, Visual Studio commands, and the short-build-root fallback for unusually deep checkouts.

For CUDA (Linux + NVIDIA GPU):

# requires the CUDA toolkit (nvcc) on PATH
cmake -B build -DTRANSCRIBE_CUDA=ON
cmake --build build

For HIP/ROCm (Linux + AMD GPU), which needs ROCm 6.1 or newer:

cmake -B build -DTRANSCRIBE_HIP=ON -DAMDGPU_TARGETS=gfx1201
cmake --build build

Replace gfx1201 with your GPU architecture — rocminfo | grep gfx prints it. Pass a semicolon-separated list for several architectures. ROCm devices report as the rocm backend kind, are picked up by the default auto backend, and can be required explicitly with --backend rocm.

libopenblas-dev is optional but recommended. It accelerates the host-side decoder ~10-15x. Without it the build falls back to a scalar path automatically.

tinyBLAS (Justine Tunney's llamafile_sgemm kernels) is on by default.

To build the quantization tool:

cmake -B build -DTRANSCRIBE_BUILD_TOOLS=ON
cmake --build build

Models

Pre-built GGUFs for all supported models are hosted on Hugging Face under handy-computer. Each per-model doc (linked in the table above) includes direct download links for every quant. Convert from source only if you need a different dtype or a checkpoint that isn't pre-built.

Convert to GGUF

The converter loads directly from NVIDIA's NeMo checkpoints via ASRModel.from_pretrained. Requires uv; the parakeet env ships NeMo and its deps.

uv run --project scripts/envs/parakeet \
  scripts/convert-parakeet.py nvidia/parakeet-tdt-0.6b-v2

This writes models/parakeet-tdt-0.6b-v2/parakeet-tdt-0.6b-v2-F32.gguf following the llama.cpp-style <slug>-<QUANT>.gguf naming convention. Pass a local .nemo path or extracted directory for offline conversion.

Quantize

The transcribe-quantize tool produces smaller models from the reference GGUF. Available presets: F16, Q8_0, Q6_K, Q5_K_M, Q4_K_M.

build/bin/transcribe-quantize \
  models/parakeet-tdt-0.6b-v2/parakeet-tdt-0.6b-v2-F32.gguf \
  models/parakeet-tdt-0.6b-v2/parakeet-tdt-0.6b-v2-Q4_K_M.gguf \
  --quant Q4_K_M

Usage

build/bin/transcribe-cli -m models/parakeet-tdt-0.6b-v2/parakeet-tdt-0.6b-v2-F32.gguf samples/jfk.wav

Input must be 16 kHz mono WAV. Use ffmpeg or sox to convert other formats:

ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav

On the CPU and on Metal (Apple silicon), model weights are memory-mapped from the GGUF file rather than copied into process memory: they live in the page cache, which the OS can reclaim and share between processes, and loading is faster. Don't truncate or rewrite a model file in place while a program has it loaded; write a new file and rename it over the old one. --no-mmap (transcribe_model_load_params::no_mmap, or TRANSCRIBE_NO_MMAP=1) copies the weights instead.

Models whose FFN gate and up weights are stored back to back (every packed variant: granite, canary-qwen, qwen3-asr, voxtral, voxtral-realtime, moss, funasr-nano) are fused at load time by pointing one tensor at those bytes instead of packing a second copy, so the fusion usually costs no extra weight memory. It falls back to the copying path when the device layout does not allow that (tensor storage padded or the mapping split, e.g. a file larger than the device's max buffer size). TRANSCRIBE_NO_GATE_UP_ALIAS=1 forces the copying path.

Bindings

Official bindings wrap the C API for other languages:

See docs/bindings.md for how the bindings are generated and kept in sync with the header. Upgrading from 0.1? Read the 0.2 migration guide, including the new exact-device selection API and the changed meaning of CLI --device 0.

Tests

cd build && ctest

Some tests require a real model file. Enable them with:

cmake -B build -DTRANSCRIBE_BUILD_REAL_MODEL_TESTS=ON
cmake --build build
TRANSCRIBE_PARAKEET_GGUF=path/to/model.gguf ctest --test-dir build

For the model-family smoke-test, numerical-validation, and benchmark pattern expected of new ports, see docs/model-family-testing.md.

Sponsors & Supporting Organizations

Mozilla AI & BiR Program

A huge thanks to Mozilla AI and their BiR Program. This whole project started out as an idea, not even an implementation direction. It was a research project in how to accelerate transcription models across all platforms as easily as possible. The BiR program and Davide helped support the research, and my eventual direction to choose to implement and inference engine backed by ggml. And also experimenting with automated model porting using agentic programming tools.

Hugging Face

Hugging Face provided the project extra storage so we can host all of the models which we support. We want to provide canonical references for as many models as reasonably possible, the support from Hugging Face helps to enable this.

Modal

Modal helped to provide GPU credits so the project can test and validate the projects implementations match the transformers or nemo reference source. This is critical to ensuring that we have as close to a production grade inference engine that works everywhere. We believe it is critical to have accurate transcriptions and the only way to ensure this is through long running WER checks which Modal helps to provide. Every model published under handy-computer on hugggingface has had the WER checked, so you can trust the results. And if there are any regressions, you bet we will be fixing them.

Blacksmith

Blacksmith provides many of the CI runners for this project. That helps to keep transcribe.cpp well tested and ensure our releases are as smooth as possible. The CI is quick and a drop in replacement for the standard Github Actions runners. I ran into limits very fast with them and super happy upon reaching out to Blacksmith they were able to provide runners for the project.

Project layout

include/transcribe.h       Public C API (single header)
src/                       Library internals (C++17)
src/arch/parakeet/         Parakeet family implementation
src/arch/cohere/           Cohere Transcribe family implementation
examples/cli/              CLI binary source
tools/transcribe-quantize/ Quantization tool source
bindings/                  Python, TypeScript, Rust, and Swift bindings
docs/                      Porting and validation guidance
scripts/                   Python converter + test tooling
ggml/                      Vendored ggml (see ggml/UPSTREAM for its recipe)
patches/ggml/              Downstream patches applied by scripts/sync-ggml.sh
src/third_party/miniz/     Vendored miniz deflate codec (see its UPSTREAM file)
samples/                   Test audio files
tests/                     Unit and smoke tests

License

transcribe.cpp is MIT-licensed. See LICENSE for details. Vendored third-party components (ggml, miniz — both MIT) are attributed in THIRD-PARTY-LICENSES.md.