Idea: optional application-layer security for exposed servers (attack screening, automatic bans, per-route policies) with fastapi-guard

[rennf93]

WhisperLiveKit is often run where other people can reach it: a GPU box on a LAN, a demo, a server behind a tunnel. Every session it accepts costs real compute (a decoder per WebSocket, model inference per upload), and the access control today is the optional API token (basic_server.py:24-42). Without a token the server is open; with one, nothing tracks a client that keeps presenting wrong tokens, sends attack payloads, or opens session after session.

I would like to propose an opt-in integration of fastapi-guard, an application-layer security engine that I maintain. It runs inside the app, so it knows WhisperLiveKit's routes, and it complements whatever reverse proxy or firewall sits in front (TLS stays there). Off by default; nothing changes unless it is turned on.

What it would add:

• Attack-pattern screening of the path, query string and headers (path traversal, injection, XSS, scanner probes, with semantic analysis), on the HTTP routes and on the WebSocket handshake of /asr and /v1/listen, plus automatic bans for clients that keep probing. A banned client is refused at the handshake, before any decoder is started.
• Per-route policies: a per-client budget on new WebSocket sessions and on POST /v1/audio/transcriptions, size and content-type limits on uploads, an optional IP allowlist for private deployments, and a behavior rule that bans a client after repeated 401s on the HTTP API.
• IP and CIDR allow and block lists, country allow or block, blocking cloud-provider IP ranges, and user-agent filtering, all enforced at the WebSocket handshake as well.
• Emergency mode, which locks the server down to an allowlist during an incident, and a passive (log-only) mode to try everything before enforcing it.
• Optionally, rules changed at runtime without a restart and security events for monitoring, through a separate agent that is off by default.

On #455: a per-client session budget and bans limit how fast one client can open sessions and shut out abusers, but they do not cap concurrent sessions or close idle ones. Those two belong in the server itself, and this would complement them, not replace them.

How I would wire it, in one small PR if this direction is welcome:

• A config option (and CLI flag) such as --security-guard, off by default. When it is off, guard is never imported and create_app is unchanged.
• The middleware is added in create_app (basic_server.py:631-641) inside CORSMiddleware, so blocked responses still carry CORS headers, and the WebSocket routes get the guard's handshake dependency.
• The client address: main() passes forwarded_allow_ips to Uvicorn (basic_server.py:679-680), which rewrites the client from X-Forwarded-For before any middleware runs. The guard would trust exactly the same proxies, and with the guard on and no proxy configured Uvicorn's proxy headers are turned off, so a client cannot rotate its identity to dodge a budget or a ban.
• Request bodies (audio) are never scanned, and the free-text context query parameter is excluded from screening.
• State lives in memory, so nothing new has to be deployed.
• Tests through real WebSocket handshakes and real HTTP requests: off means no middleware; attack payloads get refused; the session budget and bans work at the handshake; a spoofed X-Forwarded-For from an untrusted peer is ignored; normal streaming is unaffected.

Would this be useful for the project, and would you prefer it as a built-in option or as a documented recipe?

Disclosure: I maintain fastapi-guard and guard-core, the engine underneath it. Both are MIT. I used AI assistance to prepare this and reviewed and verified everything myself.

Project background: Guard Core's first library is fastapi-guard, public since August 2024 (first PyPI release, 0.1.0, on 2024-08-26). It has 825 GitHub stars, 65 releases and about 62,000 PyPI downloads in the last month (pypistats.org). The framework-agnostic engines and the adapters for other frameworks and languages came later and run the same detection rules, checked in CI against shared conformance fixtures.