Repository navigation
Home
A security-first proxy in front of LLM providers: an OpenAI-compatible API with a firewall, injection scoring, PII detection, response sanitisation, an audit ledger, and cost controls.
The documentation site is the reference, and it is large: a guide, an API reference, eight security pages, plugin documentation and a SOC section. This wiki does not duplicate it. It answers the operator's questions that are spread across all of those: what is actually caught, what to settle before production, and what changes when you run more than one process.
Current version: 1.32.1.
- Threat coverage: which layer catches what, the measured OWASP numbers with their caveats, and what is deliberately out of scope
- Before production: the defaults to change, and the one API call that can turn the defences off
- Scaling and state: what is per process, what is shared, and why that changes the meaning of your rate limits
- FAQ: short answers with pointers into the reference
Requests arrive on an OpenAI-compatible API, pass an ASGI firewall and a chain of analysers, get routed to a provider, and the response is sanitised on the way back. Every mutation is written to a hash-chained audit ledger.
Two things that follow from that and are worth knowing on day one:
It is on the critical path. When the proxy is unhealthy, LLM traffic stops. Plan the restart and the monitoring accordingly.
The defences are configuration, not compile-time guarantees. The firewall and the authentication can both be turned off by a setting, which is convenient in development and is the thing to lock down in production. See Before production.
LLMProxy · README · Documentation · Report a vulnerability privately
Operating it
Reference
Project