AISG v1.1.0 — Streaming Chat Completions

What's New

Streaming Chat Completions (SSE)

Requests with stream: true are now fully supported. The gateway scans the full request through DLP before forwarding, then streams the LLM response back as Server-Sent Events — matching the OpenAI streaming spec.

• Drop-in compatible — set stream: true in your existing OpenAI SDK calls, no other changes needed
• DLP on every request — PII redaction and prompt injection blocking run on the full request before streaming begins
• Token usage included — stream_options.include_usage is automatically enabled; usage appears in the final chunk
• Graceful error handling — mid-stream errors return a structured error chunk followed by data: [DONE]

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="your-key")

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")
    
    

---

What Changed

Added

• Full SSE streaming support for chat completions
• OpenAI-compatible streamed responses
• Streaming passthrough while preserving request-side enforcement

---

What Does NOT Change

• Request-side DLP (PII redaction, prompt injection blocking) — identical behavior
• Provider resolution and routing — same logic
• API key auth, rate limiting, fail-closed security — all unchanged
• Response-side DLP scanning — not in scope (planned for a future release)

---

Files Changed

File	Change
proxy-api/app/providers.py	Added forward_chat_completion_stream()
proxy-api/app/routers/proxy.py	Removed stream: true rejection, added SSE streaming path
proxy-api/tests/test_proxy.py	Updated test to verify streaming is accepted

---

Links

Documentation

• Managed Cloud — 1M free credits, no credit card

PR

• PR #1

Full Changelog

v1.0.0...v1.1.0

---

Release Notes

When publishing:

• Set as Latest release
• Enable Create a discussion for this release

1.0.0

AISG v1.0.0 — Production Release

The AI Security Gateway is production-ready. This release marks the first stable version of the self-hosted AI firewall and LLM proxy.

Core Features

AI Firewall (DLP)

• 13 PII entity types detected and redacted via Microsoft Presidio — emails, phone numbers, credit cards, SSNs, names, locations, IP addresses
• 5 secret recognizers — OpenAI/Anthropic/GCP API keys, AWS access keys, RSA/EC private keys, GitHub tokens (PAT, OAuth), Slack webhooks
• Prompt injection blocking — detects and blocks jailbreak attempts, instruction overrides, system prompt extraction, DAN attacks, developer mode exploits
• Configurable policy: redact (replace with [REDACTED]) or block (reject the request entirely)
• Adjustable confidence threshold per deployment

LLM Proxy

• OpenAI SDK compatible — drop-in replacement, change one line of code
• Multi-provider routing via LiteLLM (Groq, OpenAI, and any LiteLLM-supported provider)
• Header-based provider and model override (x-provider, x-model)
• Structured aisg_metadata in every response (provider selected, latency, PII findings, action taken)

Security

• Fail-closed by default — if Presidio is unreachable, requests are blocked, never forwarded unscanned
• API key authentication enabled out of the box
• Token bucket rate limiting per API key (default 10 req/sec)
• Structured logs with automatic secret scrubbing
• Zero telemetry — no external calls, no analytics, no phone-home

Deployment

• Single docker compose up — gateway + Presidio orchestrated with health checks
• Environment variable resolution in gateway.yaml (${VAR} / ${VAR:-default})
• Configurable via 3 files: gateway.yaml, providers.json, .env

What's Next

See the roadmap for upcoming features.

The managed cloud version extends this with 28+ PII entity types, OCR image scanning, smart cost routing across 8+ providers, automatic failover, per-project budget enforcement, dashboards, and analytics — 1M free credits, no credit card required.

Links

• Documentation
• Managed Cloud
• OSS vs Cloud comparison

---

Apache 2.0 — Copyright 2026 Datum Fuse LLC