Skip to content

LLMIngress v0.1.0-beta.2

Pre-release
Pre-release

Choose a tag to compare

View all tags
@IamNotShady IamNotShady released this 30 Jul 03:04
· 74 commits to main since this release
v0.1.0-beta.2
54077a2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Second beta of LLMIngress — a self-hosted LLM gateway and console.

Highlights

  • 22 new provider templates (13 → 35 built-in)
    • Subscription (OAuth): Grok and MiniMax Coding Plan join Claude Code and OpenAI Codex. MiniMax Coding Plan introduces the device-code flow: show a user code, open the verification page, poll to completion — no redirect URI needed
    • API key: Anthropic, AWS Bedrock, BytePlus ModelArk, Cerebras, ClinePass, Command Code, Fireworks AI, GLM Coding Plan, Groq, Kimi Coding Plan, Mistral, Mistral Vibe, NousResearch, NVIDIA NIM, Ollama Cloud, OpenAI, OpenCode Go, Qwen Token Plan, Xiaomi MiMo, Xiaomi MiMo Token Plan
  • Upstream quota tracking: providers that report subscription windows, monthly budgets, or token plans get a periodic Worker quota probe; the Console shows remaining quota per connection
  • Gateway circuit breaker: per-connection retries with exponential backoff, plus an error-rate breaker (sliding window, half-open recovery) that short-circuits failing connections before they burn requests
  • Cross-catalog model metadata fallback: models missing capability or price data in one catalog source now fall back to the others (models.dev, OpenRouter, LiteLLM, Vercel), with expanded price-sync coverage and prefix-vendor resolution
  • Console UI redesign: refreshed layout and theming across all pages, dialogs, and drawers
  • Deploy improvements: branch-scoped Compose projects (isolated containers, network, and PostgreSQL volume per branch), automatic repair of stale container-network state, and one unified env-file precedence (shell → .env.local → .env → defaults) across every startup path

Deployment

Two-container Docker Compose delivery (one app container + PostgreSQL 18.4):

git clone https://github.com/IamNotShady/LLMIngress.git
cd LLMIngress
git checkout v0.1.0-beta.2
./scripts/deploy.sh

deploy.sh generates a random ENCRYPTION_KEY into .env when missing, then starts Compose. The Console listens on http://127.0.0.1:3000 and the Gateway on http://127.0.0.1:4000; all ports bind to loopback only by default.

⚠️ Back up the ENCRYPTION_KEY in .env — stored provider credentials can only be decrypted with the same key.

Upgrading from v0.1.0-beta.1

Check out the new tag and re-run ./scripts/deploy.sh with the same .env. The app container applies the two new migrations (0002_provider_quota, 0003_provider_oauth_device) automatically on start. Keep the same ENCRYPTION_KEY.

Note: on the main branch the Compose project name is unchanged (llmingress), so existing containers and the PostgreSQL volume carry over.

Known limitations (beta)

  • PostgreSQL uses the Compose-bundled password and its port is published to 127.0.0.1 only; adjust the Compose file if you need external access
  • /v1/embeddings is not supported in this release
Assets 2
Loading