Skip to content
73 changes: 51 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,48 +1,75 @@
# SimpleL7Proxy

> Self-hosted Layer-7 AI gateway for Azure — priority queuing, async orchestration, and per-user governance inside your own VNET.

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![.NET](https://img.shields.io/badge/.NET-10-purple)](https://dotnet.microsoft.com)
[![.NET 10](https://img.shields.io/badge/.NET-10-purple)](https://dotnet.microsoft.com)
[![Platform](https://img.shields.io/badge/platform-Azure%20Container%20Apps-0078D4)](https://learn.microsoft.com/en-us/azure/container-apps/overview)
[![Build](https://img.shields.io/badge/build-passing-brightgreen)](docs/DEVELOPMENT.md)

**TL;DR**
- **Run locally:** `git clone … && dotnet run --project src/SimpleL7Proxy`
- **Deploy to ACA:** `./.azure/setup.sh && azd provision && ./.azure/deploy.sh`
- **Use async mode** for long LLM calls (>60 s); see [AsyncOperation.md](docs/AsyncOperation.md)

---

![SimpleL7Proxy routes client requests through a priority queue to multiple Azure OpenAI backends, with health checking and circuit breaking on each backend.](docs/arch.png)

*Incoming requests are priority-queued and dispatched to healthy backends; degraded backends are isolated automatically.*

An open-source, self-hosted **AI Gateway** for Azure. SimpleL7Proxy is a high-performance Layer 7 router purpose-built for LLM workloads — providing priority queuing, fair-share governance, circuit breaking, async orchestration, and streaming token telemetry, all running inside your own VNET.
---

→ **[Full overview, architecture, and use-case analysis](docs/OVERVIEW.md)**
## Key Capabilities

![Architecture Diagram](docs/arch.png)
- **Priority queuing** — routes high-priority users ahead of batch traffic.
- **Per-user validation** — blocks callers whose model or header values aren't in their allowlist.
- **Entra App ID gating** — unknown app IDs rejected at the gate; no backend hit.
- **Circuit breaker** — progressive back-off; auto-recovery when backends respond.
- **Async orchestration** — blob + Service Bus hand-off for calls that exceed the sync timeout.
- **Hot-reload config** — allowlists, routing rules, and profiles update without restart.

→ **[Full architecture and use-case analysis](docs/OVERVIEW.md)**

---

## Prerequisites

- [.NET 10 SDK](https://dotnet.microsoft.com/download)
- [Docker](https://docs.docker.com/get-docker/) (for container builds)
- [Azure Developer CLI (azd)](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) (for cloud deployment)
- An Azure subscription with Container Apps and (optionally) AI Foundry / APIM
- [Docker](https://docs.docker.com/get-docker/) (container builds)
- [Azure Developer CLI (azd)](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) (cloud deployment)
- Azure subscription with Container Apps; optionally AI Foundry / APIM

## Quick Start

**Local development** (interactive setup wizard):
**Local (2 commands):**
```bash
git clone https://github.com/your-org/SimpleL7Proxy.git
cd SimpleL7Proxy
dotnet run --project src/SimpleL7Proxy
```

**Deploy to Azure Container Apps:**
```bash
# Windows
./.azure/setup.ps1

# Linux / macOS
chmod +x ./.azure/setup.sh && ./.azure/setup.sh

**Azure Container Apps — Windows:**
```powershell
.\.azure\setup.ps1
azd provision
./.azure/deploy.ps1 # or deploy.sh on Linux/macOS
.\.azure\deploy.ps1
```

See [Development and Testing](docs/DEVELOPMENT.md) for local mock backends and manual configuration.
See [Container Deployment](docs/CONTAINER_DEPLOYMENT.md) for all deployment scenarios (standard, high-performance, VNET).
**Azure Container Apps — Linux / macOS:**
```bash
chmod +x .azure/setup.sh .azure/deploy.sh
./.azure/setup.sh && azd provision && ./.azure/deploy.sh
```

> See [Development & Testing](docs/DEVELOPMENT.md) for local mock backends.
> See [Container Deployment](docs/CONTAINER_DEPLOYMENT.md) for VNET and high-performance variants.

---

## Documentation

**New here?** Start with [Quick Start](#quick-start) → [Overview](docs/OVERVIEW.md) → [Advanced Configuration](docs/ADVANCED_CONFIGURATION.md).

| Topic | Document |
|-------|----------|
| Overview & Architecture | [docs/OVERVIEW.md](docs/OVERVIEW.md) |
Expand All @@ -56,18 +83,20 @@ See [Container Deployment](docs/CONTAINER_DEPLOYMENT.md) for all deployment scen
| Request Validation | [docs/REQUEST_VALIDATION.md](docs/REQUEST_VALIDATION.md) |
| Observability & Telemetry | [docs/OBSERVABILITY.md](docs/OBSERVABILITY.md) |
| Security | [docs/SECURITY.md](docs/SECURITY.md) |
| Environment Variables | [docs/ENVIRONMENT_VARIABLES.md](docs/ENVIRONMENT_VARIABLES.md) |
| Configuration Settings | [docs/CONFIGURATION_SETTINGS.md](docs/CONFIGURATION_SETTINGS.md) |
| Azure App Configuration | [docs/AZURE_APP_CONFIGURATION.md](docs/AZURE_APP_CONFIGURATION.md) |
| Environment Variables | [docs/ENVIRONMENT_VARIABLES.md](docs/ENVIRONMENT_VARIABLES.md) |
| AI Foundry Integration | [docs/AI_FOUNDRY_INTEGRATION.md](docs/AI_FOUNDRY_INTEGRATION.md) |
| APIM Policy | [APIM-Policy/readme.md](APIM-Policy/readme.md) |
| Container Deployment | [docs/CONTAINER_DEPLOYMENT.md](docs/CONTAINER_DEPLOYMENT.md) |
| Development & Testing | [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) |
| Response Codes | [docs/RESPONSE_CODES.md](docs/RESPONSE_CODES.md) |

---

## Contributing

Issues and pull requests are welcome. Please open an issue first to discuss significant changes.
Issues and pull requests are welcome. **Open an issue first** to discuss significant changes before submitting a PR.

## License

Expand Down
Loading