rust-genai

Providers and adapters can also be forced with a namespaced model name via [name_space]::[model_name], for example, qwen_cloud::qwen3.8-max

Custom OpenAI-compatible endpoints can be accessed with the built-in genai_n adapter. Set GENAI_{n}_ENDPOINT and optionally GENAI_{n}_API_KEY, then use the genai_{n}::model_name namespace:

GENAI_1_ENDPOINT=https://my-host.example.com/v1/
GENAI_1_API_KEY=sk-your-api-key

let client = Client::new()?;
let chat_res = client.exec_chat("genai_1::my-model", chat_req, None).await?;

For an OMLX deployment, set OMLX_ENDPOINT to the OpenAI-compatible base URL before creating the client. OMLX uses OMLX_API_KEY when authentication is required:

OMLX_ENDPOINT=http://my-omlx-host:8000/v1/
OMLX_API_KEY=your-api-key

Then target the model with the omlx:: namespace:

let client = Client::new()?;
let chat_res = client.exec_chat("omlx::my-model", chat_req, None).await?;

Also supports custom endpoints and auth with ServiceTargetResolver (see examples/c06-target-resolver.rs) to support any other providers.

// Can talk to any models / providers
let client = Client::new()?;

let question = "Why is the sky red?";

let chat_req = ChatRequest::new(vec![
	ChatMessage::system("Answer in one sentence"),
	ChatMessage::user(question),
]);

// Model names can even have a reasoning effort suffix, such as "-high", which will be set, and then removed from name when sent to the provider.
let chat_res = client.exec_chat("gpt-5.4-mini-high", chat_req, None).await?;

println!("{}", chat_res.first_text().unwrap_or("NO ANSWER"));

Docs for LLMs | CHANGELOG | BIG THANKS

Key Features

• Multi-AI provider/model access optimized per provider: native protocols when available, OpenAI-compatible APIs when appropriate or required, and one common Rust API for OpenAI, OpenAI Responses, Anthropic, Gemini, Ollama, Ollama Cloud, OpenCode Go, Groq, xAI, DeepSeek, Cohere, Together, Fireworks, Nebius, Mimo, Zai, BigModel, Aliyun, Atlas Cloud, Google Vertex, GitHub Copilot, Mistral, Requesty, API Route, Qwen Cloud, Kimi, and OMLX (direct chat and streaming), plus custom genai_{n} OpenAI-compatible endpoints (see examples/c00-readme.rs)
• Image analysis (for OpenAI, Gemini Flash-2, Anthropic) (see examples/c07-image.rs)
• Custom auth/API key (see examples/c02-auth.rs)
• Model aliases (see examples/c03-mapper.rs)
• Custom endpoint, auth, and model identifier (see examples/c06-target-resolver.rs)
• And much more

Examples | Thanks | Library Focus | Changelog | Provider Mapping: ChatOptions | Usage

Model to Adapter Resolution

By default, the library resolves the AdapterKind (AI provider) based on the model name prefix:

• OpenAI: gpt-* (most), o1-*, o3-*, o4-*, chatgpt-*, codex-*
• OpenAI Responses: gpt-5-*, gpt-6-*, gpt-* (containing codex or pro)
• Anthropic: claude-*
• Gemini: gemini-*
• Gemini Interactions API (Argon): gemini-4-argon* models are routed to the gemini_ix adapter. The gemini_ix:: namespace also forces this adapter.
• xAI: grok-*
• DeepSeek: deepseek-*
• Moonshot: moonshot-*
• Kimi: kimi* (moonshot.ai)
• Zai: glm-*
• Cohere: command-*, embed-*
• Mimo: mimo-*
• OpenCode Go: Namespace opencode_go:: only
• Mistral AI: Namespace mistral:: only
• Atlas Cloud: Namespace atlascloud:: only
• Qwen Cloud: Namespace qwen_cloud:: only, for the managed Qwen Cloud service at qwencloud.com. It uses the international Aliyun-compatible endpoint and QWEN_CLOUD_API_KEY.
• Fireworks: Models containing fireworks
• Ollama: Fallback for any other names, defaulting to local Ollama.

Namespacing (Forcing an Adapter)

You can force a specific adapter by using the adapter_kind::model_name syntax. This is the recommended way for many providers and for disambiguating OpenAI-compatible services.

• groq::openai/gpt-oss-20b (Forces Groq adapter)
• mistral::mistral-large-4 (Forces Mistral AI adapter)
• together::meta-llama/Llama-3-8b-chat-hf (Forces Together adapter)
• fireworks::glm-5p1 (for fireworks.ai)
• ollama_cloud::gemma3:4b (Forces Ollama Cloud adapter)
• github_copilot::openai/gpt-5.4-mini (Forces GitHub Copilot adapter)
• nebius::Qwen/Qwen3-235B-A22B (Forces Nebius adapter)
• aliyun::qwen-plus (Forces Aliyun adapter)
• qwen_cloud::qwen-plus (Forces Qwen Cloud adapter, managed by qwencloud.com and backed by the international Aliyun-compatible service)
• vertex::gemini-3.8-flash (Forces Google Vertex adapter)
• gemini_ix::gemini-3.5-flash (Forces Gemini Interactions API adapter)
• moonshot::moonshot-v1-8k (Forces Moonshot adapter)
• baidu::ernie-4.0 (Forces Baidu adapter)
• zai_coding::glm-4.6 (Special namespace for Zai coding subscription)
• opencode_go::minimax-m2.5 (Forces OpenCode Go adapter)
• bedrock_api::anthropic.claude-v2 (Forces AWS Bedrock adapter)
• open_router::google/gemini-2.0-flash-001 (Forces OpenRouter adapter)
• requesty::openai/gpt-4o-mini (Forces Requesty adapter)
• api_route::gpt-6.1-sol or api_route::claude-fable-5-1 (Forces API Route adapter)
• heabsy::qwen38 (Forces Heabsy adapter)
• runcrate::deepseek-ai/DeepSeek-V4.1-Flash (Forces Runcrate adapter)
• cloudgpu::deepseek-v4.1-flash (Forces CloudGPU adapter)
• cheapaimodel::deepseek-v4-1-flash (Forces cheapaimodel adapter)
• atlascloud::qwen/qwen3.5-flash (Forces Atlas Cloud adapter)
• genai_1::my-model-7b (Forces Custom adapter with index 1)

For a complete list of AdapterKind, see the AdapterKind enum.

v0.7.0 Released

v0.7.0 release date: 2026-10-19

See all in CHANGELOG and v0.7.0 migration guide for more details.

Big release: Many fixes, new capabilities, new providers, and minimal API changes.

Key Breaking Changes:

• Fallible client initialization: Client::new()? and ClientBuilder::build()? return Result<Client> (no more Client::default()).
• ReasoningEffort::Zero replaces ReasoningEffort::None ("zero" keyword, "none" backward-compatible alias).
• Error::HttpError now carries the failed response headers.
• Tool adds custom_format, and ChatOptions adds raw_frame_sink, stream_header_timeout, and stream_read_timeout (use ..Default::default() or add the fields to struct literals).
• Optional None fields are omitted from serialized JSON instead of null.
• JsonSpec::schema_with_additional_properties_false is removed; adapters now sanitize schemas.

New Providers & Adapters:

• Gemini Interactions API (gemini_ix::, and gemini-4-argon* models mapped to it): stateful conversation sessions (previous_response_id, store).
• Mistral AI (mistral::): Chat Completions with typed thinking content, assistant reasoning replay, and random_seed mapping.
• Atlas Cloud (atlascloud::), Qwen Cloud (qwen_cloud::), Kimi (kimi::, kimi-*), OMLX (omlx::, omlx-*).
• Custom adapter (genai_{n}::): generic OpenAI-compatible routing via GENAI_{n}_ENDPOINT and GENAI_{n}_API_KEY.

Key Capabilities & Improvements:

• Declarative provider config via ClientBuilder::append_provider_config and ClientConfig::append_provider_config.
• Stream frame inspection via ChatFrameSink and ChatOptions::with_raw_frame_sink.
• Thought signature and signed thinking block preservation across tool turns (Anthropic, Gemini, OpenRouter).
• Bedrock Converse: preserve streaming usage from metadata after the stop event, replay signed and redacted reasoning across tool turns, and refine publisher detection, Nova/OpenAI reasoning mappings, and document naming.
• Anthropic prompt caching on tools (Tool::with_cache_control) and SSE heartbeat events (ChatStreamEvent::Heartbeat).
• OpenTelemetry GenAI semantic conventions instrumentation behind the otel feature flag.

Examples

• examples/c00-readme.rs - Quick overview code with multiple providers and streaming.
• examples/c01-conv.rs - Shows how to build a conversation flow.
• examples/c02-auth.rs - Demonstrates how to provide a custom AuthResolver to supply auth data, such as api_key, per adapter kind.
• examples/c03-mapper.rs - Demonstrates how to use a custom ModelMapper to customize model mapping.
• examples/c04-chat-options.rs - Demonstrates how to set chat generation options such as temperature and max_tokens at the client level, for all requests, and at the per-request level.
• examples/c05-model-names.rs - Shows how to get model names per AdapterKind.
• examples/c06-target-resolver.rs - For custom auth, endpoint, and model.
• examples/c07-image.rs - Image analysis support

• genai ModelMapper code demo (v0.1.7)

• genai introduction (v0.1.0)

• genai live coding, code design, and best practices

  • Adding Gemini Structured Output (vid-0060)
  • Adding OpenAI Structured Output (vid-0059)
  • Splitting the JSON value extension trait into its own public crate, value-ext value-ext
  • (part 1/3) Module, Error, constructors/builders
  • (part 2/3) Extension Traits, Project Files, Versioning
  • (part 3/3) When to Async? Project Files, Versioning strategy

Library Focus:

• Focuses on standardizing chat completion APIs across major AI providers while preserving provider-specific strengths.

• Native implementation without per-service SDK dependencies.

  • Reason: genai uses each provider's native protocol when available, so features such as reasoning controls, thinking budgets, streaming metadata, and multimodal inputs can be represented more completely. When a provider primarily exposes an OpenAI-compatible API, genai uses that compatibility layer instead. Managing these protocol differences at the adapter layer is simpler and more scalable than dealing with multiple SDKs.

• Prioritizes ergonomics and commonality, while depth is secondary. (If you require a complete client API, consider using async-openai and ollama-rs; both are excellent and easy to use.)

• This library focuses on text chat, vision, and function calling APIs. (If you require a complete client API, consider using async-openai and ollama-rs; both are excellent and easy to use.)

ChatOptions

• (1) - OpenAI-compatible notes
  • Models: OpenAI, DeepSeek, Groq, xAI, Mimo, Together, Fireworks, Nebius, Zai, AIHubMix, Atlas Cloud

Usage

• (1) - OpenAI-compatible notes

  • Models: OpenAI, DeepSeek, Groq, xAI, Mimo, AIHubMix, Atlas Cloud
  • For Groq, the property is x_groq.usage.
  • Usage availability can vary by provider and streaming mode.
  • prompt_tokens_details and completion_tokens_details will have the value sent by the compatible provider, or None

• (2): Gemini tokens

  • Right now, with the Gemini Stream API, it’s not clear whether usage for each event is cumulative or must be summed. It appears to be cumulative, meaning the last message shows the total number of input, output, and total tokens, so that is the current assumption. See possible tweet answer for more info.

Usage examples

• AIPack - Check out AIPack, which wraps this genai library into an agentic runtime to run, build, and share AI Agent Packs. See pro@coder for a simple example of how I use AI PACK/genai for production coding.

• zcoder - I am also in the process of building zcoder, which will be a parallel-first coding harness.

Note: Feel free to send me a short description and a link to your application or library that uses genai. I'm happy to add it.

TLS Backends

TLS is selectable via cargo features. The default works out of the box and is right for almost everyone.

rustls-tls and native-tls are mutually exclusive. When selecting native-tls, set default-features = false (as shown above) — otherwise both backends compile and the build fails with a compile_error! telling you to do so.

rustls-tls reads the OS trust store via rustls-platform-verifier, so enterprise/custom CAs work without native-tls, and it needs no system OpenSSL. Note the default aws-lc-rs provider builds aws-lc-sys (a C/assembly crate); for a leaner static-musl or cross build, drop it with default-features = false and use the ring provider as in the table above.

For full control, build a reqwest::Client yourself and inject it:

let reqwest_client = reqwest::Client::builder()
    // .add_root_certificate(...), client identity for mTLS, custom CryptoProvider, etc.
    .build()?;

let client = genai::Client::builder()
    .with_reqwest(reqwest_client)
    .build()?;

If you set default-features = false on genai without enabling a TLS feature, add reqwest with a TLS feature directly to your own Cargo.toml (or use the rustls-no-provider path and install a CryptoProvider) — otherwise HTTPS requests fail at runtime.

The default system-proxy feature turns on reqwest's system-proxy, which reads the macOS and Windows system proxy settings when a client is built. Cargo unifies features, so it applies to every reqwest client in the binary. With default-features = false it is off; add features = ["system-proxy"] to keep it. Proxy environment variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY) are honored either way.

Contribution

Contributions are welcome. To keep review fast and the codebase consistent, please follow these guidelines.

• Keep your PR focused on one problem or one enhancement, or on very closely related changes. Small, focused PRs are easier to review and faster to merge.

• This crate follows a zero-panic approach, so avoid .unwrap(), .expect(...), and similar panic-prone calls in the production code path, and avoid array indexing with [] as much as possible.

  • For now, .unwrap() and .expect(...) are tolerated in tests and examples, but they should be avoided going forward and will be changed soon. Favor .ok_or("...")? and other non-panicking alternatives.

• When changing a provider or adapter implementation behavior, updating or adding a Yakbak cassette is appreciated when it makes sense.

• Using AI is completely fine, as long as you fully review and control the end result before submitting the PR.

  • Concise, to-the-point description.
  • Clean and focused code.
  • Emoticonless, especially in code.

• Do not update CHANGELOG.md, BIG-THANKS.md, or the migration docs under docs/migration/ in your PR. The maintainer handles those in a single pass so the formatting, wording, and ordering stay consistent across releases. Describe the change in your PR instead, and call out anything user-visible or breaking so it can be captured accurately.

  • If CHANGELOG.md or migration files are erroneous, submitting a PR to rectify them is a good approach.

Links

• crates.io: crates.io/crates/genai
• GitHub: github.com/jeremychone/rust-genai