XHub

XHub is an MIT-licensed, self-hosted AI gateway for teams using multiple model providers. Give applications a shared endpoint and separate virtual keys, manage model access and budgets, and inspect usage through a web console. Platform teams manage shared AI resources while business teams connect applications within their assigned permissions.

Distinct XHub capabilities: Compose enterprise pre-call checks in XGo, inherit routing templates through key/team/organization scopes, continue agent calls with Responses history, retain per-request price snapshots, and deduplicate settlement for supported asynchronous media tasks. Model catalogs, Playground, and call tracing connect these capabilities to everyday use.

Website and online documentation · Static website deployment

As AI adoption grows from one application to multiple departments, model access needs a shared management layer.

Manage provider credentials, deployments, public model names, and supported endpoints together. Dialogue endpoints cover OpenAI Chat/Responses, Anthropic Messages, and Gemini/Vertex, with conversion between supported dialogue structures and registered native forwarding paths. Applications use agreed model names while platform teams manage the deployments behind them.

Provider discovery and catalogs help administrators configure models and let business users inspect available models, capabilities, and prices. Available operations depend on the provider and deployment capabilities.

Organize resources into organizations, teams, and projects, with platform, organization, and team administrators alongside ordinary members. Platform teams set model and budget boundaries; organization and team administrators manage members and projects within their scope. Members get their own model catalog and call records.

Give people personal keys and applications team or project service keys. Keys support model scopes, budgets, expiration, rotation, blocking, and deletion. Removing a member preserves personal keys; subsequent requests must pass current membership, key scope, and billing authorization checks. Model permissions narrow through these scopes, with management and data access checked by the backend.

Review usage and calculated costs by user, key, team, and project. Configure budgets and RPM/TPM allocations on organization, team, user, and API Key pages. Pricing supports catalog and manual rates, input/output and cache read/write distinctions, token, picture, second, and query units, and peak/off-peak windows.

Each call retains its applied price snapshot, so later price changes preserve historical pricing evidence. Gateway response caching and provider prompt caching are recorded separately to support usage and cost reconciliation. Available measurement dimensions depend on the upstream data reported.

Configure weighted traffic splits and deployment selection based on load, cost, latency, or usage through model settings and routing templates. Organize models into routing groups and define fallback chains for general errors, context limits, and upstream content-policy errors. Combine them with retries, timeouts, and failure cooldowns.

Organizations, teams, and keys can select or inherit routing templates. Session affinity and complete response caching for non-streaming calls are supported. Cross-model fallback happens only before response bytes are sent and can be disabled with disable_fallbacks:true; pinned continuations, native asynchronous creation, and Responses background requests are not replayed on another model.

Apply keyword, RE2 regular-expression, secret-detection, and custom XGo checks to applicable Chat/Responses text. Rules can run by default or on demand, in priority order, with blocking, redaction, modification, or non-blocking flags and records of actual execution.

XGo guardrails can combine HTTP, JSON, and LLM calls for business checks. Implemented adapters connect OpenAI Moderation, Lakera, Azure, Bedrock, Presidio, and remote LiteLLM guardrails. The console supports draft testing, configuration validation, and rule management. The current execution stage is pre-call; see the guardrail guide for coverage.

Handle multi-turn messages and tool-call structures in supported dialogue protocols. Continue Responses history through previous_response_id so subsequent turns can send incremental input. Associate sessions, call IDs, keys, and usage to trace consecutive requests from business assistants and coding agents.

The Playground supports streaming and non-streaming calls, session continuation, and complete curl examples to validate real requests and results before application integration.

Connect image generation/editing, speech generation/transcription, embeddings, reranking, and other operations according to declared deployment capabilities and their implementations. Video task transports include Volcengine Ark, Qiniu Seedance, and registered Fal queues. Protocol, model, and pricing support depends on the deployment.

Supported asynchronous tasks can be created and queried, with links between the original request, terminal results, and usage. Logs show running, completed, or failed states; repeated completion queries deduplicate settlement. See the runtime reference and Qiniu Fal extension for provider-specific operations and pricing boundaries.

Manage credentials, deployments, access, routing, guardrails, and usage through the English and Chinese console. Validate models and endpoints in the Playground, diagnose missing credentials or unavailable deployments, inspect request and error logs for failure stages, upstream responses, and pricing evidence, and review key administrative actions through audit records.

Self-host the Go gateway, web console, and data stores. Your organization manages the runtime environment and access policies. Health checks help investigate service and dependency state; model requests go to the provider endpoints you configure.

1. Connect providers. Add credentials and deployments, publish model names, and configure routing.
2. Create organizations and teams. Assign administrators, add members, and set team model access and budgets. Create projects as needed.
3. Issue keys for people and applications. Use personal keys for members and service keys for team or project applications.
4. Validate in the Playground, then integrate. Check available models and responses before connecting applications to the gateway API.
5. Manage adoption over time. Review usage and cost, adjust deployments and routing, revoke access, and investigate problems through logs.

flowchart LR
    Apps[Business apps / Internal tools / AI agents] -->|API + virtual key| Gateway[XHub Enterprise AI Gateway]
    Platform[Platform and team admins] --> Console[XHub Console]
    Console -->|Configuration / Access / Usage / Testing| Gateway
    Gateway -->|Route and forward| Providers[Configured AI providers]
    Gateway --- DB[(PostgreSQL)]
    Gateway --- Redis[(Redis · optional)]

The gateway listens on 4000 and the console on 3000 by default. PostgreSQL stores configuration, identities, and usage. Redis provides shared runtime state; it is optional when running from source and included in the Compose stack.

A model catalog for business teams. The model page shown above displays the models available to the current account, their capabilities and prices, with access to Playground testing.

A shared configuration workspace for platform teams. Manage provider credentials, deployments, and public model names in Models + Endpoints.

Screenshots show a configured local instance. A fresh installation starts without model deployments.

Start with one team and one application, then expand along your organizational structure.

Install Docker Desktop on macOS/Windows, or Docker Engine and the Compose plugin on Linux. Start Docker and check:

docker version
docker compose version

The current deployment builds on your host and packages the output into Docker images. You also need Git, Go 1.25, Node.js ≥24.14.1, npm ≥11.10.0, and a readable system CA bundle. The build script detects the common Debian/Ubuntu, RHEL, macOS, and Alpine paths; set XHUB_CA_CERT_FILE for another location. The build script requires Bash; on Windows, install these tools inside WSL2 Linux and enable Docker Desktop WSL integration. The script supports amd64/arm64 and downloads Linux Node.js from npmmirror. See the installation guide for complete requirements.

git clone https://github.com/qiniu/xhub.git
cd xhub
bash deploy/build.sh
docker compose up -d --no-build
docker compose ps

The build script creates local xhub-gateway and xhub-console images. Compose starts the gateway, console, PostgreSQL, and Redis. PostgreSQL data is stored in the xhub-pg named volume. Use a fresh database for your first installation.

The initial local administrator is admin@xhub.local / admin-pass-1234. Change the password after signing in, then:

1. Add provider credentials, a deployment, and a public model name in Models + Endpoints.
2. Create an organization and team, set model access and budgets, and add members. Create projects as needed.
3. Issue personal keys for members and service keys for applications.
4. Test in the Playground, then connect applications using the gateway URL and virtual keys.

A fresh installation starts without model deployments. Follow the user and administration guide for complete steps.

# Follow gateway and console logs
docker compose logs -f gateway console

# Restart application services
docker compose restart gateway console

# Stop services and retain the PostgreSQL volume
docker compose down

# Start again
docker compose up -d --no-build

Back up your database before updating. From the repository directory, pull the new version, rebuild, and recreate the application containers:

git pull --ff-only
bash deploy/build.sh
docker compose up -d --no-build --force-recreate gateway console

For a server deployment, set NEXT_PUBLIC_BASE_URL to the browser-accessible gateway URL before building, and set Compose's XHUB_PUBLIC_ORIGIN to the same public API URL. Keep the console container's XHUB_GATEWAY_ORIGIN set to a reachable internal gateway address. Gateway settings, including initial administrator credentials, are in configs/config.docker.yaml; rebuild images after changing them. Initial password configuration only creates accounts that do not exist; change an existing account's password in the console. See deployment configuration for details.

For local development or independently managed PostgreSQL/Redis, follow the source installation guide to start the Go gateway and Next.js console separately. The gateway and console still use ports 4000 and 3000; applications connect the same way.

Install the client with pip install openai. Use your XHub virtual key and the public model name you configured:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_XHUB_VIRTUAL_KEY",
    base_url="http://localhost:4000/v1",
)

response = client.chat.completions.create(
    model="YOUR_PUBLIC_MODEL_NAME",
    messages=[{"role": "user", "content": "Hello, XHub!"}],
)
print(response.choices[0].message.content)

SDK requests go to the gateway on port 4000; port 3000 serves the web console. Check the configured deployment's capabilities before integrating other operations.

Installation and customer guides are available in English and Chinese. Central development references are in Chinese; package-level readme.md / readme_cn.md files provide bilingual code guides.

XHub is actively evolving. Evaluate production requirements against the runtime reference, permission rules, and guardrail guide. Budgets do not reserve request costs; shared limits, caching, and idempotency have implementation boundaries across instances. Reconcile calculated costs with provider bills. Enterprise identity supports independently managed OIDC Code + PKCE and SAML 2.0 login, plus SCIM user/group provisioning; enterprise login requires an active provisioned identity. Real identity-provider tenants require separate validation; this repository does not promise commercial SLAs or compliance certifications.

XHub is licensed under the MIT License, allowing commercial use, modification, and redistribution subject to retaining the copyright and permission notice. Third-party dependencies and incorporated materials retain their own licenses and notices, including provider catalog attribution. The software is provided without warranty.

XHub is actively evolving. Open an issue with an enterprise use case or reproduction steps, improve the documentation, or submit a pull request. For code changes, follow the developer guide and the relevant module's validation requirements.

Budgets follow Organization → Team → Person → API Key, in USD. Each person belongs to at most one team. Fixed child budgets reserve their unused amounts; sibling allocations and consumed money must fit within the parent budget. Blank budgets share the parent's unreserved balance; zero blocks spending. An organization budget of $1,000 can allocate $500 + $300 + $200 across teams, but cannot allocate another dollar. Personal keys without an explicit team binding still bill the owner's sole team. Service keys bill a separate business account. Use a separate unlimited organization and team for unrestricted business usage.

RPM and TPM follow the same hierarchy: team allocations sum within the organization, personal allocations within the team, and key allocations within the person. Fixed values reserve minute capacity; blank values share the unreserved remainder; zero blocks calls. Blank intermediates pass fixed descendant allocations upward. Over-allocation rolls back the save. Admission atomically counts the unique ownership path and rejected calls consume no rate capacity. TPM uses the incoming token estimate; windows are UTC calendar minutes. Multiple gateway instances need shared Redis.

See quota management for examples, administrator permissions, membership changes, historical spending, and the request-admission/settlement boundary. For code ownership, allocation invariants, admission and accounting flow, and layered test commands, see quota implementation and verification. Quota acceptance uses fresh temporary ownership chains for each budget and RPM/TPM check, verifies rejection before any upstream call and successful billed recovery, then deletes all temporary resources. Existing allocations and spending remain intact. The actual acceptance script is covered by a local gateway/database regression test; see acceptance dataset documentation.