AI Proxy

A production-ready AI gateway proxy with PII protection, multi-provider support, and comprehensive request logging.

✨ Features

🔐 PII Protection: Automatic detection and blocking of sensitive data (API keys, passwords, private keys, JWT tokens, AWS secrets)
🔄 Multi-Provider Support: OpenAI, Anthropic, GitHub Copilot with unified API
📊 Request Logging: Track all requests, responses, tokens, and latency
⚖️ Load Balancing: Round-robin distribution across multiple API keys
🔑 API Key Management: Secure storage with AES-256-GCM encryption
👤 OAuth Authentication: GitHub and Google OAuth support
📈 Dashboard: Web UI for monitoring and management
💾 MySQL Database: Production-ready with automatic indexing
🚀 Easy Deployment: Docker + GitHub Actions CI/CD

🏗️ Architecture

  ┌──────────────────────┐          ┌─────────────────────────────┐
  │  Browser / Dashboard │          │  API Clients (curl/SDK/IDE) │
  │  Next.js (Port 3000) │          │  X-Proxy-Key: rpk_...       │
  └──────────┬───────────┘          └──────────────┬──────────────┘
             │ JWT (Bearer eyJ...)                  │ Proxy Key (rpk_...)
             ▼                                      ▼
┌────────────────────────────────────────────────────────────────────┐
│                     AI PROXY  –  Rust / Axum                       │
│                                                                    │
│  ┌──────────────────┐  ┌────────────────────┐  ┌────────────────┐ │
│  │  Public Routes   │  │  Dashboard Routes  │  │  Proxy Routes  │ │
│  │  (no auth)       │  │  /api/*            │  │  /v1/*         │ │
│  │                  │  │                    │  │                │ │
│  │  OAuth callbacks │  │  JWT Middleware     │  │  ProxyKey      │ │
│  │  /health         │  │  validate + inject  │  │  Middleware    │ │
│  │                  │  │  Claims extension   │  │  SHA-256 hash  │ │
│  └──────────────────┘  │                    │  │  DB lookup     │ │
│                        │  Dashboard API      │  │                │ │
│                        │  logs / stats /     │  │  Provider      │ │
│                        │  rules / keys /     │  │  Handler       │ │
│                        │  usage / pricing    │  │  openai /      │ │
│                        └────────────────────┘  │  anthropic /   │ │
│                                                │  copilot /     │ │
│                                                │  unified       │ │
│                                                └───────┬────────┘ │
│                                                        │          │
│                          ┌─────────────────────────────▼────────┐ │
│                          │           Rule Engine                │ │
│                          │                                      │ │
│                          │  ┌──────────┐  ┌──────────────────┐  │ │
│                          │  │ Built-in │  │  DB Rules (GRL)  │  │ │
│                          │  │ patterns │  │  user-defined    │  │ │
│                          │  │ API keys │  │  regex patterns  │  │ │
│                          │  │ password │  │  custom actions  │  │ │
│                          │  │ SSH keys │  └──────────────────┘  │ │
│                          │  │ JWT/AWS  │                        │ │
│                          │  └──────────┘                        │ │
│                          │                                      │ │
│                          │   ALLOW  ──►  forward to provider    │ │
│                          │   BLOCK  ──►  return 400 to client   │ │
│                          │   REPLACE──►  redact + forward       │ │
│                          └──────────────────────────────────────┘ │
│                                                        │          │
│  ┌─────────────────────────────────────────────────────▼────────┐ │
│  │                    Storage Layer  (MySQL)                    │ │
│  │                                                              │ │
│  │  users  │  provider_keys (AES-256-GCM)  │  user_proxy_keys  │ │
│  │  request_logs  │  rules  │  usage_summaries  │  pricing      │ │
│  └──────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
                                  │
          ┌───────────────────────┼────────────────────┐
          ▼                       ▼                    ▼
  ┌───────────────┐      ┌────────────────┐    ┌──────────────────┐
  │  OpenAI API   │      │ Anthropic API  │    │  Copilot API     │
  │  GPT-4o / o1  │      │ Claude Sonnet  │    │  GitHub Models   │
  └───────────────┘      └────────────────┘    └──────────────────┘

  ┌──────────────────────────────────────────────────────────────┐
  │  Cost Worker  (background / standalone binary)               │
  │                                                              │
  │  runs every N seconds                                        │
  │  ──────────────────────                                      │
  │  recalculate_usage()  ──►  request_logs  ──►  usage_summaries│
  │  estimate cost per model from pricing config                 │
  └──────────────────────────────────────────────────────────────┘

Request Flow

API Client                   AI Proxy                    LLM Provider
    │                            │                            │
    │── POST /v1/llm/chat ──────►│                            │
    │   X-Proxy-Key: rpk_...     │                            │
    │                            │── validate rpk_ key ──►DB  │
    │                            │◄─ user_id ─────────────────│
    │                            │                            │
    │                            │── Rule Engine ────────────►│
    │                            │   scan body for PII        │
    │                            │◄─ ALLOW / BLOCK / REPLACE ─│
    │                            │                            │
    │◄── 400 if BLOCKED ─────────│                            │
    │                            │── GET provider key ───►DB  │
    │                            │── forward request ────────►│
    │                            │◄─ response ────────────────│
    │                            │── log + cost calc ────►DB  │
    │◄── response ───────────────│                            │

Auth Flow

Dashboard User                AI Proxy              GitHub / Google
    │                             │                       │
    │── GET /api/auth/oauth/conf ►│                       │
    │◄── { client_id, redirect } ─│                       │
    │                             │                       │
    │── redirect to provider ────────────────────────────►│
    │◄── callback ?code=... ──────────────────────────────│
    │                             │                       │
    │── GET /api/auth/google/cb ─►│                       │
    │             code=...        │── exchange code ──────►│
    │                             │◄─ access_token ────────│
    │                             │── GET /userinfo ──────►│
    │                             │◄─ email, name, id ─────│
    │                             │── upsert user ────►DB  │
    │◄── { token: "eyJ..." } ─────│                       │
    │                             │                       │
    │── POST /api/* ─────────────►│                       │
    │   Authorization: Bearer eyJ │── validate JWT ───────►│
    │                             │   inject Claims ext.   │

🚀 Quick Start

Prerequisites

Rust 1.75+ (curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh)
Node.js 18+ (brew install node or download from nodejs.org)
MySQL 8.0+

Backend Setup

cd ai-proxy

# Copy and edit environment config
cp .env.example .env
nano .env

# Generate encryption key
openssl rand -hex 32

# Run the proxy
cargo run --release

Frontend Setup

cd dashboard

# Install dependencies
npm install

# Run development server
npm run dev

Services will be available at:

Dashboard: http://localhost:3000
API Proxy: http://localhost:8080

📖 Configuration

All configuration is done via environment variables (.env file in ai-proxy/):

# Server
SERVER_HOST=0.0.0.0
SERVER_PORT=8080

# Database (MySQL required)
DATABASE_TYPE=mysql
DATABASE_URL=mysql://user:password@localhost:3306/aiproxy

# Security — generate with: openssl rand -hex 32
ENCRYPTION_KEY=your-64-char-hex-key

# Retention
RETENTION_DAYS=30

# OAuth / JWT — generate with: openssl rand -base64 32
JWT_SECRET=your-jwt-secret
JWT_EXPIRY_HOURS=24

# GitHub OAuth
GITHUB_CLIENT_ID=your_github_client_id
GITHUB_CLIENT_SECRET=your_github_client_secret
GITHUB_REDIRECT_URI=https://your-domain.com/auth/callback/github

# Google OAuth
GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret
GOOGLE_REDIRECT_URI=https://your-domain.com/auth/callback/google

See ai-proxy/.env.example for the full list with comments.

📚 Documentation

MYSQL.md: MySQL setup and schema
DEPLOYMENT.md: Complete deployment guide with GitHub Actions
SECURITY.md: Security features and encryption details

🔌 API Usage

Provider Routing

Code-Based Routing

# Route by provider code
curl -X POST https://llm-gateway-api.ironcode.cloud/v1/unified/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-Proxy-Key: rpk_your_proxy_key" \
  -H "X-Provider-Code: oai-prod" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Auto-Detection

# Automatically detect provider from model name
curl -X POST https://llm-gateway-api.ironcode.cloud/v1/unified/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-Proxy-Key: rpk_your_proxy_key" \
  -d '{
    "model": "claude-3-opus-20240229",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Direct Provider APIs

# OpenAI
curl -H "X-Proxy-Key: rpk_your_proxy_key" \
  https://llm-gateway-api.ironcode.cloud/v1/openai/chat/completions

# Anthropic
curl -H "X-Proxy-Key: rpk_your_proxy_key" \
  https://llm-gateway-api.ironcode.cloud/v1/anthropic/messages

# GitHub Copilot
curl -H "X-Proxy-Key: rpk_your_proxy_key" \
  https://llm-gateway-api.ironcode.cloud/v1/copilot/chat/completions

IDE & Tool Integration

Replace your existing API key with a proxy key (rpk_...) and point your tool at the gateway:

opencode

// .opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "iron-gateway": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Iron LLM Gateway",
      "options": {
        "baseURL": "https://llm-gateway-api.ironcode.cloud/v1/llm",
        "apiKey": "rpk_your_proxy_key"
      },
      "models": {
        "gpt-4o": { "name": "gpt-4o" },
        "claude-sonnet-4-6": { "name": "claude-sonnet-4-6" }
      }
    }
  }
}

Continue (VS Code / JetBrains)

// ~/.continue/config.json
{
  "models": [
    {
      "title": "GPT-4o (Gateway)",
      "provider": "openai",
      "model": "gpt-4o",
      "apiKey": "rpk_your_proxy_key",
      "apiBase": "https://llm-gateway-api.ironcode.cloud/v1/openai"
    },
    {
      "title": "Claude Sonnet (Gateway)",
      "provider": "anthropic",
      "model": "claude-sonnet-4-6",
      "apiKey": "rpk_your_proxy_key",
      "apiBase": "https://llm-gateway-api.ironcode.cloud/v1/anthropic"
    }
  ]
}

Claude Code

export ANTHROPIC_API_KEY=rpk_your_proxy_key
export ANTHROPIC_BASE_URL=https://llm-gateway-api.ironcode.cloud/v1/anthropic

Any SDK (.env)

OPENAI_BASE_URL=https://llm-gateway-api.ironcode.cloud/v1/openai
OPENAI_API_KEY=rpk_your_proxy_key
ANTHROPIC_BASE_URL=https://llm-gateway-api.ironcode.cloud/v1/anthropic
ANTHROPIC_API_KEY=rpk_your_proxy_key

🛡️ Security Features

PII Detection

Automatically blocks or redacts:

API keys (OpenAI, Anthropic, AWS, GitHub, etc.)
Passwords and credentials
Private keys (RSA, SSH, PGP)
JWT tokens
AWS access keys and secrets

API Key Encryption

All provider API keys stored in the database are encrypted with AES-256-GCM:

Argon2 key derivation from ENCRYPTION_KEY
Random nonce per encryption
Keys never exposed through the proxy

OAuth Authentication

GitHub OAuth integration
Google OAuth integration
JWT-based session management

📊 Features in Detail

Load Balancing

Configure multiple API keys per provider for automatic round-robin distribution:

# Add primary key
curl -X POST https://llm-gateway-api.ironcode.cloud/api/keys \
  -H "Authorization: Bearer eyJ..." \
  -H "Content-Type: application/json" \
  -d '{"provider": "openai", "name": "Primary Key", "api_key": "sk-...", "code": "oai-prod"}'

# Add backup key (same code = same load-balance group)
curl -X POST https://llm-gateway-api.ironcode.cloud/api/keys \
  -H "Authorization: Bearer eyJ..." \
  -H "Content-Type: application/json" \
  -d '{"provider": "openai", "name": "Backup Key", "api_key": "sk-...", "code": "oai-prod"}'

Request Logging

All requests are logged with:

Timestamp and latency
Provider and model
Input/output tokens
Request/response bodies
PII violations
Client IP and user agent

GitHub Copilot (Device Flow)

To add a Copilot key, the gateway uses GitHub's device flow (no redirect required):

Go to /keys → Add Key → select copilot
A one-time code is displayed — visit github.com/login/device and enter it
The gateway polls GitHub and saves the access token linked to your account

🔧 Development

Project Structure

rust-llm/
├── ai-proxy/               # Rust backend
│   ├── src/
│   │   ├── main.rs        # Entry point
│   │   ├── config.rs      # Configuration (env vars)
│   │   ├── auth/          # OAuth + JWT authentication
│   │   ├── proxy/         # Provider proxies (openai, anthropic, copilot, unified)
│   │   ├── rules/         # Rule engine & PII detection
│   │   ├── server/        # Axum routes & middleware
│   │   ├── storage/       # MySQL storage layer
│   │   ├── dashboard/     # Dashboard API handlers
│   │   ├── pricing.rs     # Cost estimation (65+ models)
│   │   └── encryption.rs  # AES-256-GCM encryption
│   ├── src/bin/
│   │   └── worker.rs      # Standalone cost worker binary
│   ├── Cargo.toml
│   └── .env.example
├── dashboard/             # Next.js frontend
│   ├── src/
│   │   └── app/          # App router pages
│   ├── package.json
│   └── next.config.ts
└── migrations/            # MySQL schema migrations

Build Commands

Rust

cd ai-proxy
cargo build                # Debug build
cargo build --release      # Release build
cargo run --release        # Run proxy
cargo run --bin worker     # Run cost worker
cargo test                 # Run tests
cargo clippy               # Lint
cargo fmt                  # Format

Next.js

cd dashboard
npm install                # Install deps
npm run dev                # Development
npm run build              # Production build
npm start                  # Start prod server
npm run lint               # Lint

🚀 Production Deployment

Automated Deployment with GitHub Actions

Configure GitHub Secrets:
- DOCKER_USERNAME, DOCKER_PASSWORD
- SSH_HOST, SSH_USERNAME, SSH_PRIVATE_KEY
- DEPLOY_PATH
Push to main branch:
```
git push origin main
```
GitHub Actions will automatically:
- Build Docker images
- Push to Docker Hub
- Deploy to production server
- Run health checks

See DEPLOYMENT.md for complete setup instructions.

📈 Monitoring

Health Check

curl https://llm-gateway-api.ironcode.cloud/health

Logs

# Rust application logs
RUST_LOG=debug cargo run

Statistics

# Get usage stats (requires JWT)
curl -H "Authorization: Bearer eyJ..." \
  https://llm-gateway-api.ironcode.cloud/api/stats

🤝 Contributing

Contributions are welcome! Please:

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Code Style

Rust: Follow rustfmt defaults, run cargo fmt
TypeScript: Follow project ESLint rules, run npm run lint

📄 License

MIT License - see LICENSE file for details.

Made with ❤️ using Rust and TypeScript

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
.github/workflows		.github/workflows
ai-proxy		ai-proxy
dashboard		dashboard
.env.example		.env.example
.gitignore		.gitignore
AGENTS.md		AGENTS.md
LICENSE		LICENSE
README.md		README.md

Folders and files

Latest commit

History

Repository files navigation

AI Proxy

✨ Features

🏗️ Architecture

Request Flow

Auth Flow

🚀 Quick Start

Prerequisites

Backend Setup

Frontend Setup

📖 Configuration

📚 Documentation

🔌 API Usage

Provider Routing

Code-Based Routing

Auto-Detection

Direct Provider APIs

IDE & Tool Integration

opencode

Continue (VS Code / JetBrains)

Claude Code

Any SDK (.env)

🛡️ Security Features

PII Detection

API Key Encryption

OAuth Authentication

📊 Features in Detail

Load Balancing

Request Logging

GitHub Copilot (Device Flow)

🔧 Development

Project Structure

Build Commands

Rust

Next.js

🚀 Production Deployment

Automated Deployment with GitHub Actions

📈 Monitoring

Health Check

Logs

Statistics

🤝 Contributing

Code Style

📄 License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages