Repository navigation
Before Production
From the project's own checklist, with why each one matters:
| Setting | Default | Change to | Because |
|---|---|---|---|
| TLS | disabled | enabled, or a reverse proxy in front | API keys and prompts cross the wire otherwise |
| CORS | ["*"] |
your frontend origins | Any page on the internet can otherwise call your proxy from a victim's browser |
| Auth | enabled | keep enabled, rotate keys | Leave it on. See below for how easily it comes off |
| API keys | placeholder | strong, generated | A placeholder key is a public key |
| Presidio | not installed | install it | Without it, PII detection is regex-only |
| tiktoken | not installed | install it | Without it, token counts and therefore budgets are estimates |
The proxy warns at startup when TLS is disabled or CORS is unrestricted. Read the startup log once, deliberately, on the first production boot.
POST /api/v1/config/apply rewrites config.yaml live and hot-reloads it. It requires an admin credential, and then an admin can flip server.auth.enabled: false or security.firewall.enabled: false in a single request, turning off the proxy's own authentication and its firewall. Validation warnings on those deltas are non-blocking.
This is tracked as #108, which proposes treating posture-lowering deltas as privileged, behind a stronger credential than the ordinary admin bearer.
Until that lands, three things follow:
- An admin token is equivalent to the ability to disable the proxy. Scope who holds one accordingly, and keep it out of anything that also holds a user token
- Watch for the change. A config apply that disables auth or the firewall should page someone. The audit ledger records it, which only helps if something reads the ledger
-
Consider whether
/api/v1/config/applyneeds to be reachable at all in your deployment. If configuration is managed by your deployment pipeline, a network policy in front of the admin routes is cheaper than waiting for the fix
The firewall can be turned off in configuration:
security:
firewall:
enabled: falseThat exists for development and integration tests, and it is exactly the kind of setting that reaches production inside a copied config file. Grep your deployed configuration for it before going live, and again after any config change that came from a developer's machine.
A fresh proxy boots with zero endpoints and returns 503 for inference until you add one. That is a good default, and it means a deployment that looks broken on the first request is probably just empty. Check the registry before debugging anything else.
The container keeps budget tracking, the audit log and registered endpoints in /app/data. Without a mounted volume that is all lost on restart, including the audit ledger, which is the one piece of state whose loss you cannot reconstruct.
docker run -d --name llmproxy -p 8090:8090 \
-e LLM_PROXY_API_KEYS=... \
-v llmproxy-data:/app/data \
ghcr.io/fabriziosalmi/llmproxy:1.32.1Pin the version. Every release publishes the full semver, the minor, :latest, and a per-commit SHA tag. :latest on a security component means an unreviewed change can arrive during an unrelated restart.
The proxy decides what leaves your application; it does not decide where the host can connect. For a hardened deployment, put network-level egress filtering underneath it, with domain allowlisting and direct-IP blocking, so a compromised process cannot simply talk to a provider of its own choosing. secure-proxy-manager is the sibling project for that layer.
LLMProxy · README · Documentation · Report a vulnerability privately
Operating it
Reference
Project