flowcat

Why Flowcat

Most voice-agent platforms are hosted SaaS: your audio, transcripts, and call data flow through someone else's cloud, and you pay per minute. Flowcat is the opposite — a runtime you own and run yourself, for teams that can't or won't put regulated call traffic on a multi-tenant platform and want one auditable artifact instead of a fleet of services.

1. Own your voice stack

Flowcat is a single self-contained binary you deploy in your own VPC — or fully air-gapped. There is no Flowcat cloud and no phone-home: the runtime reads only its own FLOWCAT_* config and talks to the providers you configure with your credentials. Pair it with the local STT/TTS/LLM connectors (Whisper, Kokoro / Piper / XTTS, Ollama) and a call's audio and transcript never leave your infrastructure. For data-residency, on-prem, and sovereignty requirements (healthcare, finance, public sector), that's the deployment model itself — not a checkbox bolted onto a SaaS.

2. Long-call memory — without the audio tax

Realtime models re-bill the whole conversation every turn — as bulky, expensive audio — so the usual way to keep long calls affordable, sliding-window compression, saves money by discarding the oldest turns (your agent forgets the account number from minute one). Flowcat's ContextRelay converts that accumulated audio context into a compact text transcript and reseeds the session: the model re-attends cheap text (~7× smaller, ~4× cheaper per token) instead of audio, and the whole conversation survives. The live voice path is unchanged — the agent still speaks native audio; the trade-off is that converted turns carry their words, not their prosody. Off by default, provider-agnostic. See the ContextRelay evaluation.

3. pipecat-compatible by design

If you know pipecat, you already know Flowcat. It deliberately mirrors pipecat's architecture and public API model — the FrameProcessor graph, the typed Frame taxonomy, the system-frame priority / interruption model, and the STT/TTS/LLM/realtime service seams. You bring the same mental model and the same vendor credentials; you get a single static Rust binary instead of a Python process tree. See Connectors & providers.

4. One process, room to scale

Because the media loop is Rust — no garbage collector, no GIL — one Flowcat process uses every core and holds a flat p99 from 10 to 2,000 concurrent calls on a single box, where an equivalent Python deployment grows a multi-second tail and needs a worker fleet. Read this as capacity and operational headroom, not as a claim about conversational latency: end-to-end voice latency is dominated by your STT/LLM/TTS providers (hundreds of ms) and Flowcat can't change that. What it guarantees is that the runtime itself never becomes the bottleneck or the source of a stall — so you provision fewer boxes and your tail stays predictable under load. See Benchmark & capacity.

5. Drive it from Python

You don't have to write Rust. Run Flowcat as a service and drive it from Python at turn granularity: implement your conversation policy as a small HTTP service (the RemoteBrain adapter, brain-http feature) and expose your Python functions as tools over MCP. Your code never touches the per-frame path, so the capacity profile above is preserved. In-process PyO3 bindings are on the roadmap. See Using Flowcat from Python and examples/.

Benchmark & capacity

What this measures: how much call volume one box absorbs before the runtime becomes the bottleneck, and how tight the tail stays under load. What it does not measure: end-to-end conversational latency — that's dominated by your STT/LLM/TTS providers (typically hundreds of ms), and Flowcat doesn't change it. Read the sub-millisecond figures below as framework/transport overhead — capacity and reliability headroom — not as the latency a caller hears.

A like-for-like benchmark on a single Azure Standard_FX16mds_v2 box (16 vCPU): one Flowcat process (12 cores) vs pipecat in its real multiprocess deployment (12 workers, SO_REUSEPORT, one per core — Python given every advantage). Identical Rust WebSocket + μ-law load generator, full-duplex echo, 50 frames/s/call, 10 s per data point.

p99 round-trip latency vs concurrent calls

Flowcat's line is flat along the floor; pipecat crosses the ~150 ms conversational limit at a few hundred calls and reaches multi-second tails by 1,000.

Other measured metrics

Full percentile distributions (p50 / p90 / p99 / p99.9 / max), the methodology, and the phase history are in bench/RESULTS.md.

Reproduce it

docker compose -f bench/compose.yml up --build   # on a 16-vCPU VM

See bench/README.md for the full harness and SKU notes.

Disclaimer. Numbers above are from the reproducible kit in this repo on the stated hardware; your results will vary with hardware and configuration. pipecat is an independent open-source project; it is used here as an architecture reference and a benchmark baseline. Flowcat is not affiliated with, sponsored by, or endorsed by Daily or the pipecat project. "pipecat" is referenced for identification and comparison only; all marks belong to their respective owners. See NOTICE.

How to use it

Flowcat is a Cargo workspace of four library crates plus a demo binary. Nothing networked is in the default build — every provider, transport, and exporter is an opt-in Cargo feature.

# Build the whole workspace (default features only → no provider client deps).
cargo build

# Run the full fixture/wire test suite (no network, no credentials).
cargo test

# Build a "fat" binary that pulls in every provider client:
cargo build -p flowcat-services \
  --features stt-all,tts-all,llm-all,realtime-all,obs-all

# The demo binary — two runnable, credential-free demos:
cargo run -p flowcat-cli -- pipeline           # in-process FrameProcessor pipeline
cargo run -p flowcat-cli -- ws-echo --loopback # real WebSocket PCM echo round-trip

Run an agent from a config — no Rust

Don't want to embed Flowcat in a Rust binary? Run flowcat-server: describe one agent in a YAML/JSON config (a node/edge graph + the realtime or cascaded provider topology) and serve it over HTTP — no control plane, no database.

cargo build --release -p flowcat-server --features webrtc
GOOGLE_API_KEY=… ./target/release/flowcat-server --config deploy/agent.example.yaml
# open http://localhost:6210/ to talk to it (mic + live transcript), or bridge a
# Plivo number to the server's /telephony/ws/plivo/{run_id}

Providers are selected by name from the config (the flowcat-services factory) and their keys come from the environment; the agent graph is run by flowcat-agent. A Dockerfile, compose file, sample config, and env template are in deploy/. The config schema lives in flowcat-server/src/config.rs.

Embed it in your own service

For full control (custom routing, your own control plane, in-process brain logic), embed the library and implement three seams:

• FrameProcessor pipeline — compose transport.input() → vad → stt → llm → tts → transport.output() (or a single realtime S2S model) into a Pipeline, drive it with a PipelineTask / PipelineRunner. Each processor runs in its own tokio task behind a bounded channel (natural backpressure).
• AgentBrain — your conversation decision-making. Flowcat never sees your control-plane, REST contract, or database; the brain is a trait seam. (Don't want to write Rust? The ready-made RemoteBrain adapter implements this seam against an HTTP service — see Using Flowcat from Python.)
• SessionSource — how a call is bootstrapped and finalized.

The runtime is provider- and contract-agnostic: it knows nothing about any downstream control plane. Full processor-author contract: PROCESSOR-DESIGN.md and CONTRIBUTING.md.

Feature-flag model

flowcat-core defaults to ["sip", "recorder"] (no HTTP/gRPC/ONNX). Every provider, transport, and exporter is dep:-gated, so adding the 80th provider costs the default build nothing. Umbrella features (stt-all, tts-all, llm-all, realtime-all, obs-all) exist for the CLI and CI. Full enumeration: FEATURES.md.

The flowcat CLI ships two demos (the analogue of pipecat's examples/), both credential-free and exercised in CI: pipeline drives a synthetic sine-wave source through a composable FrameProcessor pipeline in-process, and ws-echo echoes PCM over the real WebSocket transport (--loopback for a self-contained round-trip, or --connect <ws://url> to a live peer). See flowcat-cli/src/.

Using Flowcat from Python

Flowcat is a Rust runtime, but you don't have to write Rust to use it. The media loop (SIP/RTP, VAD, STT/LLM/TTS) runs in Rust; your Python runs at turn granularity over a network boundary, so it never sits on the per-audio-frame path that determines tail latency.

• Drive the conversation policy — the RemoteBrain adapter (flowcat-services, feature brain-http) implements the AgentBrain seam by POSTing to two JSON endpoints you host. Decide transitions, what to say, and when to end the call — in Python. Reference server + wire contract: examples/python-remote-brain.
• Expose Python functions as tools — run an MCP server; Flowcat's mcp client lists and calls its tools. examples/python-mcp-tools.

In-process PyO3 bindings — import flowcat, build a pipeline in Python, pass Python callables as the brain — are on the roadmap; they will keep Python at turn granularity to preserve the same tail-latency guarantees.

Connectors & providers

Start here: the live-verified path. The Gemini Live + Plivo combination — speech-to-speech over WebSocket-media telephony — is the one path run end-to-end against the real services today. Build on it first, and treat everything below as wire-ready but unproven until you run it yourself.

Beyond that, Flowcat carries a broad provider catalogue, each connector one dep:-gated Cargo feature so the default build pulls none of their client dependencies:

The full feature-flag matrix is in FEATURES.md; how the distinct (D) clients and thin (W) wrappers relate is in PROVIDERS.md.

What "supported" means here — read this before you count providers. Every connector is fixture/wire-tested: unit tests pin its message framing (plus SigV4 known-answer tests for the AWS path), so the encode/decode seam is correct. They are not all exercised against the live service in CI — an end-to-end call needs that vendor's credentials. Today the Gemini Live + Plivo/Zadarma path is confirmed live end-to-end; the rest are a short step away but unproven until you run them. Live-verifying a provider against its real service is one of the most useful contributions you can make — see CONTRIBUTING.md.

How it works

 carrier / WebRTC          FrameProcessor graph (each stage = 1 tokio task)
 ───────────────►  ┌──────────────────────────────────────────────────┐
   SIP / RTP       │  transport.in → vad/turn → stt → llm → tts → out  │
   WebSocket       │                    └──── or realtime S2S ────┘     │
 ◄───────────────  └──────────────────────────────────────────────────┘
                      system frames (Start/Cancel/Interruption/End)
                      jump the queue on a priority channel

• Typed Frame taxonomy (audio / text / control / system). The hot audio frame is an Arc<AudioFrame>, so each hop moves a pointer, not a buffer.
• Each processor runs in its own tokio task fed by a bounded mpsc channel, giving natural backpressure on the output media leg. The per-hop channel cost is ~0.029 µs — three orders of magnitude under the audio frame period.
• System frames jump the queue. Start / Cancel / Interruption / End / Stop ride an unbounded priority channel and invoke a processor's start() / stop() lifecycle hooks, bypassing process_frame.
• Native SIP/RTP, in-process — softswitch optional. flowcat-core speaks SIP (REGISTER + digest auth, INVITE/ACK/BYE) and hand-rolled RTP/SDP for G.711 telephony directly, so a single binary can terminate calls with no FreeSWITCH/Asterisk at all. That's a deployment choice, not a mandate — if you already run a softswitch, keep it in front and feed audio over the WebSocket media transport. (SIP-DESIGN.md.)

Design details: PROCESSOR-DESIGN.md (frozen API + latency argument) and DESIGN.md (runtime architecture + trait seams).

Crate map

flowcat/
├── flowcat-core/        # framework: Frame, FrameProcessor, Pipeline/Task/Runner,
│                        #   Observer/metrics, audio codec/resample/recorder,
│                        #   native SIP/RTP/SDP UA, Gemini Live, all trait seams
├── flowcat-services/    # every STT/TTS/LLM/realtime provider + obs exporters + MCP
│                        #   — one cargo feature each
├── flowcat-transports/  # str0m WebRTC + Opus, WebSocket, Daily, LiveKit, local
├── flowcat-telephony/   # carrier FrameSerializers (Twilio/Telnyx/Plivo/…) + DTMF
├── flowcat-agent/       # declarative graph agent — a config-driven AgentBrain
├── flowcat-server/      # config-driven single-agent server + browser playground
├── flowcat-cli/         # `flowcat` demo binary (DX / examples surface)
├── bench/               # the reproducible pipecat-vs-flowcat benchmark kit + RESULTS.md
├── bench-rs/            # standalone load-gen + framework micro-bench
├── PROCESSOR-DESIGN.md  # the frozen FrameProcessor API + latency argument
├── DESIGN.md            # runtime architecture + the trait seams
├── FEATURES.md          # the full feature-flag matrix
└── CONTRIBUTING.md      # how to add a provider + the processor-author contract

Contributing

Contributions welcome — start with CONTRIBUTING.md. Adding a provider is usually a small, self-contained PR; the processor-author contract in PROCESSOR-DESIGN.md §2.1–§2.3 is required reading.

License & attribution

Flowcat is licensed under Apache-2.0 (LICENSE). It is built to the same architecture and API model as pipecat (BSD-2-Clause, © Daily), used as a design reference; no pipecat source is vendored. Third-party provider protocols, service names, and trademarks belong to their respective owners, and their use requires your own credentials and acceptance of each vendor's terms. Full attribution: NOTICE.