Build LLM apps and agents

❄️ ChatGPT Desktop Application (Mac, Windows and Linux)

OpenAI's response format for its open-weight model series gpt-oss

Python bindings

Pure Rust PDF library for AI/RAG: structure-aware chunking, no ML, no C deps.

OpenAI API client library for Rust (unofficial)

High-performance Rust vector database core with HNSW indexing

An open-source Rust library for building AI-powered applications, inspired by the Vercel AI SDK. It provides a robust, type-safe, and easy-to-use interface for interacting with various Large Language Models (LLMs).

Awesomely fast virtual sandbox with bash and file system

actix-web transport implementations for RMCP (Rust Model Context Protocol)

Core protocol types and traits for SACP (Symposium's extensions to ACP)

Ship your code, on autopilot. An open source agent that lives on your machines 24/7 and keeps your apps running. 🦀

Rust crate to fix malformed JSON outputs from LLMs

Universal LLM API client — 174 providers, streaming, tool calling. Rust-powered, type-safe, compiled.

An unofficial Rust library for the OpenAI API.

Unofficial ChatGPT desktop app for Mac & Windows menubar using Tauri & Rust

Agent Framework for Building Autonomous Agents

A type-safe OpenRouter Rust SDK

Token-Oriented Object Notation – a token-efficient JSON alternative for LLM prompts

A Datacenter Scale Distributed Inference Serving Framework

EdgeParse core library — PDF parsing and structured data extraction

Rust client for Google Gemini API

Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors!

Multi-provider LLM abstraction library with caching, rate limiting, and cost tracking

Rust Agent Development Kit - Build AI agents in Rust with modular components for models, tools, memory, and more.

A high-performance, type-safe framework for building multi-agent workflow systems in Rust

Core library for Memvid v2, a crash-safe, deterministic, single-file AI memory.

Library for converting OpenAPI specifications to MCP tools

LangChain for Rust, the easiest way to write LLM-based programs in Rust

A library for running chains of LLMs (such as ChatGPT) in series to complete complex tasks, such as text summation.

High-performance vector database engine written in Rust

A Serde-compatible TOON (Token-Oriented Object Notation) serialization library

Structured Generation

Python bindings

A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

A Rust SDK for the Google Gemini API

OpenTelemetry components and utilities for Langfuse LLM observability

An unofficial Rust SDK for Anthropic's Claude AI API

The agent loop for Rust: stream from 7 LLM protocols, run tools, loop until done. Native or wasm32 (Cloudflare Workers).

Anthropic Rust SDK.

A high-performance Rust implementation of llama.cpp - LLM inference engine with full GGUF support

A purpose-built database engine for AI agent memory

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

Get structured, validated data out of LLMs as native Rust structs and enums. Derive a type and rstructor generates the JSON Schema, prompts the model, parses the reply, and retries on validation errors — across OpenAI, Anthropic Claude, Google Gemini, and xAI Grok.

ML framework on numr for training and inference on CPU, CUDA and WebGPU: attention, quantization, inference scheduling and distributed execution. Models include LLaMA, Mamba2, Qwen3.5, Whisper, Kokoro, VITS, VoxCPM2, Silero VAD, CLAP and super-resolution. Loads SafeTensors and GGUF.

Composable LLM agents, typed task graphs, and streaming RAG pipelines in Rust

A high-performance memory-mmap hybrid search engine built for AI, combining dense vector, sparse text, graph relations, and JSON metadata.

Python bindings

Native AI coding agent CLI - Rust port of Pi Agent

The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge

Open-Source Coding Agent in the terminal

Anthropic Rust SDK.

A simple interface for pretrained AI models

MoFA - Modular Framework for Agents. Modular, Compositional and Programmable.

A unified source of truth for LLM providers, models, pricing, and capabilities.

TUI interface for LLMs written in Rust

Mistral AI API client library for Rust (unofficial).

Fast, lightweight Firecrawl and Tavily alternative in Rust for web scraping.

TUI for serving local LLM models. Pick a model, pick a backend, serve it.

Lightweight Ollama-compatible inference server with native SafeTensors support. No Python dependencies, cross-platform WebGPU acceleration via Airframe.

Prompty is an asset class and format for LLM prompts

A fully self-contained AI agent backend framework with built-in web services, multi-LLM provider support, and comprehensive tool execution

The agent framework where every run is durable, replayable, and resumable by default. Plain async TypeScript agents on an embedded pure-Rust JS engine.

Python bindings

The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.

Putting a brain behind `cat`🐈‍⬛ Integrating language models in the Unix commands ecosystem through text streams.

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

AgentAI is a Rust library designed to simplify the creation of AI agents.

Run a team of autonomous agents on your code, right from your terminal

One Library to rule them aLLMs

Rust AI agent backend: build capabilities once, tune behavior live, serve every client

AICI: Prompts as (Wasm) Programs

elastic inference for your agentic workloads, on your terms

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

The Bleeding-Edge Production-Ready Multi-Agent Orchestration Framework in Rust

🧠 The production-ready cognitive foundation for autonomous systems such as Embodied-AI and OpenClaw. For memory management, from extraction and search to automated optimization, with SKILL, CLI, API, MCP, and insights dashboard out-of-the-box.

The autonomous, self-improving AI agent. Single Rust binary. Every channel. Recommended: the 40MB prebuilt binary for macOS, Linux and Windows: https://github.com/opencrabs/opencrabs/releases

High-performance model-routing gateway for large-scale LLM deployments

Open-source AI coding agent and agent runtime: one binary, any model, MCP-native. Runs in terminal, CI, or as a daemon — guardrails in scripts, not approval popups.

📦️ A fast, secure MCP server that extends its capabilities through WebAssembly plugins.

A protocol and modular application suite for language models.

Put a queue, deduplication and a circuit breaker in front of any LLM API, so repeated prompts cost one call and outages fail fast. Rust library, HTTP server and CLI.

LocalGPT CLI — a local-only AI assistant

Unifies multiple LLM backends in Rust.

A Rust-based terminal coding agent with modular architecture supporting multiple LLM providers

llm_client: The easiest Rust interface for local LLMs

Think before doing, prove before saving. Vestige is the local-first cognitive operating system and deterministic safety kernel for autonomous AI agents.

YantrikDB database server — multi-tenant cognitive memory with wire protocol, HTTP gateway, replication, auto-failover, and at-rest encryption

Local-first, real-time, edge-to-cloud hybrid database for multi-modal workloads

and 1 related project

A high-performance AI Gateway written in Rust, providing OpenAI-compatible APIs with intelligent routing, load balancing, and enterprise features

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

Open source framework to vibecode and prototype voice agents with Gradium APIs

A QUIC-based runtime for AI-LLM tool routing and serverless execution

A rust mlx server, ui, and model router for fast, dependency free inference on apple architecture

Run, Build, Share your AI Packs

Cut AI context cost without trusting the compressor. Every reduction is reversible, byte-exact recoverable, and carries an auditable receipt. Local-first, works through proxy, MCP, SDK, or agent wrapper.

Rust-native, library-first coding-agent runtime and terminal CLI inspired by Pi, with composable LLM providers, tools, sessions, and plugin ABI

git-like rag pipeline

A local AI chat app powered by 🦀Rust, 🧊iced, 🤗Hugging Face, and 🦙llama.cpp

Rust SDK for the EdgeQuake RAG platform

All-in-one platform for search, recommendations, RAG, and analytics offered via API

Inference server for LLMs, speech and embeddings: OpenAI-compatible routes with streaming, plus speech synthesis (TTS) and transcription (ASR). Serves Llama, Mistral, Qwen, Mamba2, MLA+MoE and hybrid models from SafeTensors, AWQ, GPTQ and GGUF.

High-performance LLM backends proxy with per-user fair-share queuing, round-robin scheduling, and a real-time TUI dashboard. Built in Rust.

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

🦀 A Pure Rust Framework For Building AGIs.

GraphRAG - Knowledge Graph RAG: meta-crate that bundles graphrag-core and graphrag-cli

Pure Rust 1-bit LLM inference engine for PrismML Bonsai models — umbrella crate

Pure Rust implementation of LLaMA-family of models, executable

Persistent cognitive memory for AI agents and robots — Hebbian learning, knowledge graph, spatial recall. Zenoh/ROS2 native. Single binary, runs offline.

Python bindings

Vision infrastructure to turn complex documents into LLM-ready data

High performance embedded vector database

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

An open protocol that gives AI agents a secure, sandboxed place to run code and use tools — with lean context usage. AXP works alongside MCP

Experimenting with LLMs through Ollama and retrieval augmented generation (RAG) in Rust

Using Large Language Models for Repo-wide Type Prediction