Skip to content
Fabrizio Salmi edited this page Sep 6, 2026 · 1 revision

FAQ

Is it a drop-in replacement for the OpenAI API?

Yes, on port 8090, once an endpoint is configured. Point your client's base URL at the proxy and use a proxy API key as the bearer token. A fresh install boots in onboarding mode with no endpoints and returns 503 for inference until you add one.

Which providers work?

The commercial ones plus anything OpenAI-compatible: LM Studio, vLLM, TGI, Ollama and similar can be declared directly in .env without editing YAML. Local endpoints with no authentication are supported by leaving the key blank.

Does it inspect responses, or only requests?

Both. Requests go through the firewall and the analysers; responses go through sanitisation, which covers entropy anomalies, steganography such as bidi overrides and zero-width characters, and prompt leaks. That is a difference from aidlp, which is egress only.

How much does it slow things down?

The proxy stack costs single-digit milliseconds at p50 on an idle process. Against an LLM call measured in seconds, that is noise. See Scaling and state for the measured numbers and their conditions.

My rate limit is not being enforced the way I set it

Rate limits and circuit breakers are per process, in memory, by design. With four workers, a limit of 100 admits up to 400. See Scaling and state.

PII is not being detected

Presidio is not installed by default, so detection falls back to regex: email, phone, SSN, credit card, IBAN, IP and API keys, and nothing else. Install presidio-analyzer and presidio-anonymizer for the NLP entity types.

A legitimate prompt was blocked

The measured benign false-positive rate is 6%, and the known tripping case is meta-discussion of attacks: asking the model to explain how prompt injection works looks like prompt injection. That is deliberate. Full results and gaps are in docs/OWASP_LLM_COVERAGE.md.

Can I turn the firewall off?

Yes, security.firewall.enabled: false, and that is what it is for in development. Make sure it does not travel to production in a copied config, and read Before production for the related concern: an admin can disable both the firewall and authentication in one API call.

Does it cover the OWASP LLM Top 10?

Three of the ten, thoroughly and with published numbers: prompt injection, sensitive information disclosure and system prompt leakage. The other seven are out of scope for a proxy because they live at build time, training time, in the caller or in the model, and the project documents them as not applicable rather than claiming them.

Where does my data live?

In /app/data: registered endpoints, budgets and the audit ledger, in SQLite by default and swappable for Postgres. Mount it as a volume or you lose the audit ledger on every restart.

How do I know the response was not tampered with?

HMAC-SHA256 response signing proves the response was not modified after leaving the proxy. Verify it client-side if that property matters to you; the signature is only useful to someone who checks it.

What authentication is supported for callers?

API keys, OIDC and JWT with Google, Microsoft and Apple, mTLS, and Tailscale zero-trust, with four RBAC roles: admin, operator, user and viewer. Note what an admin can do, in Before production.

How is it tested?

Eight CI jobs on every push: ruff, mypy, pip-audit, a supply-chain scan for .pth malware and blocked packages, a syntax check, the test suite behind a 65% coverage gate, mathematical invariants, and a Docker image size check. Raising coverage on the untested security paths is tracked as #107.