sipp

import { Endpoint, SippClient } from '@sipphq/sipp';

const blender = new SippClient();

// 1. Register a model and load it into a local WebGPU endpoint.
const edgeModel = await blender.models.add(['/models/llama3.gguf']);
const juice = await blender.add(
  'edge',
  Endpoint.local(edgeModel, { backend: 'webgpu' })
);

// 2. Or connect to a secure cloud gateway through the same client.
const ice = await blender.add(
  'cloud',
  Endpoint.gateway({
    target: 'production',
    baseUrl: 'https://gateway.example.com',
    authentication: { kind: 'bearer', value: '<short-lived-token>' },
  })
);

// Route the same operation to either endpoint.
const [smoothie, snowcone] = await Promise.all([
  blender.chat([{ role: 'user', content: 'Explain Sipp.' }], { endpoint: juice }).response,
  blender.chat([{ role: 'user', content: 'Create a Sipp app.' }], { endpoint: ice }).response,
]);

console.log(smoothie.text, snowcone.text);
await blender.close();

The unified SDK lets you dynamically partition and optimize complex application logic between local and cloud compute. Instead of wrestling with fragmented web runtimes, disconnected native wrappers for desktop, or custom middleware to protect API keys, you only need Sipp.

It packages a high-performance WebGPU engine, with a secure container gateway proxy into a single, neat toolkit. Future releases will focus on embedded vector memory, on-device PII masking, and automated smart routing. See Roadmap.

The sipp command below is the repository's contributor launcher; it is not an npm package or the application scaffolder. From a source checkout, use the launcher installed by the setup scripts, or call the underlying xtask directly:

sipp build wasm                  # Compile high-performance WebGPU assets
sipp run demos serve chat        # Launch a hardware-accelerated demo

# Equivalent commands without the repo launcher:
cargo xtask build wasm
cargo xtask run demos serve chat

Performance Benchmarks

Run them yourself here: benchmark.sipp.sh/benchmark

Disclaimer & Metric Notes:

• TTFT (Time to First Token): Measured in milliseconds (ms). Lower is better.
• Decode: Measured in tokens per second (tok/s). Higher is better.
• E2E Latency (End-to-End Latency): Measured in milliseconds (ms). Lower is better.
• A single maintainer run on an NVIDIA RTX 3080: 1 warm-up, then the mean of 3 measured runs. The model, prompt and output lengths, browser version, and cache state of this run were not recorded, so treat the figures as indicative. Results vary with model, hardware, and browser; run the benchmark above to measure your own setup.

Create Or Install

Create a ready-to-run browser application:

npm create @sipphq/sipp@latest my-app
cd my-app
npm install
npm run dev

npm maps that command to the public @sipphq/create-sipp initializer. The generated application depends on the browser SDK, @sipphq/sipp. The initializer requires Node.js 20.19+ or 22.12+, matching the generated app's Vite version.

For an existing application, choose the package by where that import executes, not by the framework name:

Never import @sipphq/sipp-server into a browser bundle.

# For code that runs in a web browser
npm install @sipphq/sipp

# For code that runs in Node.js
npm install @sipphq/sipp-server

# For native systems development and application embedding
cargo add sipp-rs

# For native macOS and iOS apps (source checkout on macOS)
sipp build swift

# For Python automation and data engineering pipelines
pip install sipppy

# Deploy the secure cloud gateway server instance via Docker
# (cloud gateway will be available in the future, currently building from source)
# docker pull noumena/sipp-gateway

Package Surfaces

Most developers should start with our pre-built packages when available. Swift support currently builds from a source checkout on macOS.

Quick Starts

1. Edge Quick Start (Hardware-Accelerated Client Inference)

Initialize the local engine client to execute model weights directly on the client machine's shader cores using WebGPU.

npm install @sipphq/sipp

import { Endpoint, SippClient } from '@sipphq/sipp';

const messages = [
  { role: 'system', content: 'Answer concisely.' },
  { role: 'user', content: 'Explain Sipp in one sentence.' },
];

const client = new SippClient();
const model = await client.models.add(['/models/model.gguf']);
const endpoint = await client.add(
  'default',
  Endpoint.local(model, {
    backend: 'webgpu',
    runtime: { context: { n_ctx: 2048 } },
  })
);

const run = client.chat(messages, {
  endpoint,
  maxTokens: 64,
});

console.log((await run.response).text);
await client.close();

2. Cloud Gateway Quick Start (Preemptive Cloud Proxying)

Cloud gateway clients use the same SippClient API layout. The gateway owns model paths, provider credentials, access policies, and centralized metrics; your client only needs its public target, URL, and authentication.

import { Endpoint, SippClient } from '@sipphq/sipp';

const client = new SippClient();
const endpoint = await client.add(
  'gateway',
  Endpoint.gateway({
    target: 'upstream-cluster',
    baseUrl: 'https://gateway.example.com',
    authentication: { kind: 'bearer', value: await getGatewayToken() },
  })
);

const run = client.query('Explain gateway inference.', {
  endpoint,
  maxTokens: 64,
});

console.log((await run.response).text);
await client.close();

3. Swift Quick Start (Native macOS and iOS)

Swift support requires macOS and Xcode. The build stages a local Swift package at .build/artifacts/swift/package for macOS and iOS.

sipp toolchain install rust-apple
sipp doctor --target swift
sipp build swift

import Foundation
import Sipp

let modelURL = URL(fileURLWithPath: "/path/to/model.gguf")
let client = try SippClient()
let model = try await client.models.add([modelURL])
try await client.add("local", model: model)

let run = client.chat(
    messages: [
        ChatMessage(role: .system, content: "Answer concisely."),
        ChatMessage(role: .user, content: "Explain Sipp in one sentence."),
    ],
    endpoint: "local",
    options: TextOptions(maxTokens: 64)
)

for await batch in run.tokens {
    print(batch.text, terminator: "")
}

let response = try await run.response
print(response.text)

Run the staged command-line example, open the sandboxed macOS app, or launch the iOS example in a simulator. Build the optional example artifacts first:

sipp build swift --examples

.build/artifacts/swift/examples/SippCLI chat \
  /path/to/model.gguf "Explain on-device inference"

open .build/artifacts/swift/examples/SippSandbox.app

xcrun simctl list devices available
sipp run examples ios --simulator <simulator-udid> \
  --model /path/to/model.gguf

See the Swift package guide and Swift examples for sandbox entitlements, supported operations, and device builds.

Native Web Framework Blueprints

Sipp includes native integration blueprints to handle Server-Sent Events (SSE) streaming, serverless route orchestration, and client hydration patterns out of the box.

• Next.js: App Router route handlers, Client Components, gateway proxies, and streaming.
• TanStack: TanStack Start server functions and TanStack Query patterns.
• React And Vite: Browser package setup, WASM assets, OPFS model loading, and gateway examples.

Documentation

The full documentation lives in docs/en. From a source checkout, use the repo launcher to build or serve the book:

sipp docs build
sipp docs serve

sipp docs installs required mdBook tooling when missing and configures the Mermaid assets used by the technical book. Without the launcher, use cargo xtask docs build or cargo xtask docs serve.

Technical Roadmap

Our core development trajectory is oriented around expanding the edge-cloud infrastructure for running hybrid systems, where local and cloud resources are orchestrated seamlessly.

For a detailed structural breakdown of milestones, memory architectures, and long-term research initiatives, see the full Sipp Technical Roadmap.

Maintainers & Contributors

To bootstrap the workspace, initialize a cross-platform profile, and inspect the available test suites:

source ./setup.sh
sipp doctor
sipp test list

(On Windows platforms, execute .\setup.ps1 inside PowerShell or setup.cmd via classic CMD if not using Git Bash or WSL).

The setup scripts install sipp as a repo-local alias for cargo xtask. If the launcher is not active, replace sipp with cargo xtask in any command below.

Common Build and Run Tasks

sipp build wasm
sipp run examples serve browser

sipp build node --backend cpu
node examples/node/query.mjs <model.gguf> "Explain Sipp."

sipp build python --backend cpu
python examples/python/query.py <model.gguf> "Explain Sipp."

sipp build swift   # macOS with Xcode only

sipp run demos serve chat

For thorough verification steps, consult the Source Builds Documentation and the Testing Framework Suite.

Repository Layout

• crates: The published core sipp-rs and low-level backend sipp-sys Rust crates.
• lib: High-level language package surfaces and gateway proxy toolkit.
• bindings: Native Node.js bindings, Python extensions, Swift UniFFI bindings, and browser-compiled WASM targets.
• apps: First-party user interfaces and monitoring implementations.
• examples: Small, runable framework integration blueprints.
• demos: Advanced browser sandboxes running on public package surfaces.
• tools/playground: Live browser-runtime profiling and hardware execution diagnostics.
• xtask/: Internal cargo automation engine driving build, test, and package deployment pipelines.

License

Sipp is licensed under the Apache-2.0 License. Vendored third-party dependencies preserve their respective upstream open-source licensing constraints and documentation requirements; see the third-party notices.