A unified LLM access layer written in Rust. One API for 172+ AI providers.

aimux is a Rust implementation of a unified LLM provider access layer. It collapses the HTTP APIs of every AI provider into a single dyn LanguageModel interface that anything upstream can call.

Unlike rig or langchain, aimux does not build agent loops, RAG, or orchestration — it focuses exclusively on unifying service access. That is the difference: aimux is an access layer, those are orchestration layers.

• 290+ provider modules — 11 native protocol implementations (OpenAI, Anthropic, Google, Bedrock, Vertex, Azure, Cohere, Mistral, xAI, DeepSeek, Anthropic-AWS) + 250 registry-backed OpenAI-compatible providers (unified provider(name, ...) entry, RFC-0017 phase 4) + modality-specific (speech/image/video/search).
• Unified, object-safe interface — the LanguageModel trait supports Box<dyn> so providers are interchangeable without changing call sites.
• Full multimodal — text, streaming, tool calling, embeddings, image, speech, transcription, video, reranking, files.
• Config-driven provider registry — provider-registry.json describes each of the 250 OpenAI-compatible providers (base URL, env var, profile quirks: top_k, tools, response_format, streaming usage, max_tokens key); one unified provider(name, ...) entry in every binding (RFC-0017 phase 4).
• Fast and small — Rust core, release profile tuned for binary size (lto, codegen-units=1, panic="abort", strip, opt-level="z").
• 8 language bindings from one core: Node, Python, Swift, Kotlin, Flutter, Go, Java, C.
• Hermetic tests — 2,650+ cassettes replay real API responses; no network or API keys required.

Benchmarked against the official OpenAI SDKs on the same machine, same mock server, same abstraction layer (HTTP + JSON, no orchestration). Full results and methodology in docs/PERF-RESULTS.md.

¹ vs Vercel AI SDK (not apples-to-apples — AISDK adds Zod validation, middleware, and telemetry per request).

• Rust core — reqwest connection pool, no GC, no runtime pauses.
• Zero memory growth — Python aimux RSS did not grow a single byte across 2000 requests; Node grew only 2 MB.
• Stable tail latency — no GC pauses means P99 stays flat even under CPU contention; the JS SDK's P99 spikes to 12.87 ms on a single core.
• FFI boundary is cheap — serialization is ~50% of overhead only on large payloads; in real LLM requests (3–10 s) it is <0.1%.

aimux/
├── aimux-core            # Core abstractions: LanguageModel / Provider / Message / StreamPart
├── aimux-providers       # 290+ provider implementations (250 registry-backed + native)
├── aimux-stream          # SSE / NDJSON stream parsing
├── aimux-provider-utils  # HTTP utilities: retry, backoff, error parsing, API-key loading
└── aimux-ffi             # C ABI (opaque handle + JSON + push callback) for non-native bindings

           ┌─ native path ──→ aimux-core + aimux-providers (direct Rust types + async)
bindings ──┤
           └─ C ABI path  ──→ aimux-ffi (opaque handle + JSON + push callback)

Rust (the core library):

cargo add aimux-core aimux-providers

Node.js:

npm install @arcships/aimux

import { openai, generateText } from '@arcships/aimux'

const model = await openai(process.env.OPENAI_API_KEY!, 'gpt-4o')
const result = await generateText(model, 'Explain Rust ownership in one sentence.')
console.log(result.text)

The package ships a typed wrapper (generateText / streamText) on top of the raw napi API. Need the raw JSON-string interface? Use import { openai } from '@arcships/aimux/raw'.

use aimux_core::prelude::*;
use aimux_providers::{OpenAIConfig, OpenAIProvider};

#[tokio::main]
async fn main() -> Result<(), AiMuxError> {
    let provider = OpenAIProvider::new(
        OpenAIConfig::new(std::env::var("OPENAI_API_KEY")?)
    );
    let model = provider.model("gpt-4o");

    let result = generate_text(
        &model,
        "Explain Rust ownership in one sentence.",
        GenerateTextOptions::default(),
    ).await?;

    println!("{}", result.text);
    Ok(())
}

use futures::StreamExt;

let result = stream_text(
    &model,
    "Write a haiku about Rust.",
    GenerateTextOptions::default(),
).await?;

let mut stream = result.stream;
while let Some(part) = stream.next().await {
    match part? {
        StreamPart::TextDelta { delta, .. } => print!("{}", delta),
        StreamPart::Finish { .. } => println!("\n[done]"),
        _ => {}
    }
}

// OpenAI → DeepSeek: only the provider name changes (RFC-0017 phase 4 —
// registry-backed; key read from the provider's env var)
use aimux_providers::{provider, provider_from_env, ProviderName};

// 推荐:类型化 ProviderName(IDE 补全 + 编译期检查)
let model = provider(ProviderName::Deepseek, None, "deepseek-chat", None)?;
// 字符串形式同样可用:
let model = provider_from_env("deepseek", "deepseek-chat", None)?;
// model usage is identical — it's all dyn LanguageModel

All 250 OpenAI-compatible providers are registry-backed: provider(name, ...) in every binding, with typed ProviderName (enum/union/consts per language). The retired per-provider shell types (XxxConfig/XxxProvider) are gone — see docs/API.md.

Full list: rfc/0004-provider-inventory.md.

aimux ships 8 bindings that share the same Rust core:

See bindings/README.md and the API docs.

cargo test -p aimux-providers --tests

Tests run on cassette playback — no network and no keys. See rfc/0003-test-cassette.md.

Contributions are welcome! Read CONTRIBUTING.md for the development setup, testing workflow, provider/binding conventions, and the pull request process. Please follow the Code of Conduct.

MIT