voxr

voxr can replace an ffmpeg decode step and a Python speech stack. It does not download model weights.

Not supported yet

• WebGPU: voxr has no wgpu feature.
• Multichannel AAC and HE-AAC: decode refuses them.
• Opus past two channels: decode refuses it.
• Other codecs: voxr refuses them. voxr-narrate can fall back to an ffmpeg on PATH for a file voxr refuses. Only that fallback runs a process.
• VAD on CUDA: no test or example in this repository runs Silero VAD on CUDA.

Crates

• voxr-narrate runs the speak and transcribe commands of the motion-video skill.
• voxr provides the sound review of that skill.

What works

• Rust: 1.95 or newer.
• CUDA: needs the setup in CUDA.
• Abbreviations: G2P is grapheme-to-phoneme, TTS is text-to-speech, ASR is automatic speech recognition, and VAD is voice activity detection.

Quick Start

Add the crate to your project, with the features you need:

cargo add voxr --features decode

Decode a file to 16 kHz mono and write it as a 16-bit WAV file:

use std::path::Path;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let samples = voxr::decode_audio_file_mono_at(Path::new("in.m4a"), 16_000)?;
    let wav = voxr::encode_wav_pcm16(&samples, 16_000)?;
    std::fs::write("out.wav", wav)?;
    Ok(())
}

To try transcription from a checkout of this repository:

1. Download a Whisper checkpoint in Hugging Face layout, such as openai/whisper-base. voxr does not download it.
2. Check that its generation_config.json lists alignment_heads.
3. Run the example:

cargo run --release -p voxr-narrate --features whisper --example transcribe_take -- \
  --model WHISPER_DIR AUDIO.wav

The example writes the words of each input to standard output.

Features

Every feature is off by default. A build with no features includes:

• WAV read and write
• Resamplers and tempo stretch
• Enhancement and single-sound metrics
• Sound review and pitch tracking
• ASR error rates
• TTS quality statistics that need no model

voxr features

Decode formats

Trims and downmix match ffmpeg. MP4 edit lists and Matroska codec delay come from containr.

voxr-narrate features

Models

No model weights are compiled in or shipped, and voxr fetches nothing over the network. Download each checkpoint, for example from Hugging Face, and pass its directory at run time.

The embedded data is text-processing data only: the misaki lexicons behind misaki, and the spaCy tokenizer tables behind spacy-tokenizer.

CUDA

• CUDA binaries: a build targets the GPUs on the build host. Set NUMR_CUDA_ARCH=all for a binary that runs on another machine.
• Every value: numr's docs/backends.md lists them.

Contributing

Contributions are welcome. See CONTRIBUTING.md for conventions and checks.

Questions and design discussion happen on Discord.

License

Licensed under Apache-2.0. See LICENSE.

Code ported from other projects and data voxr redistributes are listed with their licenses in crates/voxr/LICENSE-OTHERS and crates/voxr-narrate/LICENSE-OTHERS. Each embedded data set carries its own LICENSE and NOTICE beside it under crates/voxr/assets/.