SecureBridge

Secure remote access to on-premise AI models and compute resources.

Overview

SecureBridge is an open-source infrastructure project that enables secure remote access to on-premise AI models (like Ollama, local LLMs) through a cloud-based bridge without exposing your local network or sending sensitive data to external services.

Key Features

• OpenAI-Compatible API: Drop-in replacement for OpenAI API
• Firewall-Friendly: Client initiates outbound-only connections
• Privacy-First: Data never persists on the bridge server
• Fallback Support: Automatic fallback to cloud APIs (Anthropic Claude)
• Horizontal Scaling: Stateless server design
• Load Balancing: Multiple routing strategies (round-robin, least-connections, random)

Use Cases

• Self-host AI chat while providing public API access
• Cost optimization (use local GPU, fallback to cloud for overflow)
• Data privacy (process sensitive documents locally)
• Development (test against local models before production)
• Edge AI (deploy at edge locations with central management)

Quick Start

Prerequisites

• Go 1.21 or higher
• Ollama or other OpenAI-compatible local AI service
• (Optional) Anthropic API key for fallback

Installation

# Clone the repository
git clone https://github.com/yourusername/securebridge.git
cd securebridge

# Install dependencies
make install-deps

# Build binaries
make build

Running Locally

Terminal 1 - Start Ollama (or your local AI service):

ollama serve
ollama pull mixtral:8x7b

Terminal 2 - Start Bridge Server:

./bin/securebridge-server -config config/server.yaml

Terminal 3 - Start Bridge Client:

./bin/securebridge-client -config config/client.yaml

Terminal 4 - Test:

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer test-api-key-1" \
  -d '{
    "model": "mixtral:8x7b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Configuration

Server Configuration

Edit config/server.yaml:

server:
  port: 8080
  tls:
    enabled: false  # Set to true in production
    cert: /path/to/cert.pem
    key: /path/to/key.pem

auth:
  api_keys:
    - key: "your-api-key"
      name: "Client Name"
      rate_limit: 100
  appliance_keys:
    - key: "your-appliance-key"
      name: "Appliance Name"

fallback:
  enabled: true  # Enable fallback to cloud APIs
  provider: "anthropic"
  api_key: "${ANTHROPIC_API_KEY}"  # Use environment variable
  trigger: "appliance_unavailable"

routing:
  strategy: "round_robin"  # or "least_connections", "random"
  health_check_interval: 30s
  request_timeout: 60s

Client Configuration

Edit config/client.yaml:

client:
  server_url: "ws://localhost:8080/appliance/connect"
  api_key: "your-appliance-key"
  appliance_id: "unique-id"
  name: "My GPU Box"

local_api:
  url: "http://localhost:11434"  # Ollama default
  type: "openai"
  timeout: 60s

connection:
  reconnect: true
  reconnect_interval: 5s
  max_reconnect_interval: 60s
  heartbeat_interval: 30s

API Endpoints

OpenAI-Compatible Endpoints

POST /v1/chat/completions

Request body:

{
  "model": "mixtral:8x7b",
  "messages": [
    {"role": "user", "content": "Hello!"}
  ],
  "temperature": 0.7,
  "max_tokens": 2048
}

Health & Status

GET /health       # Server health status
GET /ready        # Readiness check

Admin Endpoints

GET /admin/appliances  # List connected appliances (requires auth)

Deployment

Docker

Server:

docker build -f deployments/docker/Dockerfile.server -t securebridge-server .
docker run -d -p 8080:8080 -v $(pwd)/config:/etc/securebridge securebridge-server

Client:

docker build -f deployments/docker/Dockerfile.client -t securebridge-client .
docker run -d --network host -v $(pwd)/config:/etc/securebridge securebridge-client

Systemd

See example service files in deployments/systemd/:

sudo cp deployments/systemd/securebridge-server.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable securebridge-server
sudo systemctl start securebridge-server

Kubernetes

kubectl apply -f deployments/kubernetes/

Architecture

External Clients � Bridge Server (Cloud) � Bridge Client (On-Premise) � Local AI

• Bridge Server: Stateless cloud component, exposes OpenAI-compatible API
• Bridge Client: On-premise component, connects to local AI service
• Communication: WebSocket with outbound-only connections (firewall-friendly)
• Security: TLS, API key authentication, no data persistence

Development

Building

make build              # Build both server and client
make build-server       # Build server only
make build-client       # Build client only
make build-all          # Build for all platforms

Running in Dev Mode

make run-server         # Run server from source
make run-client         # Run client from source

Testing

make test               # Run tests
make test-coverage      # Run tests with coverage

Security

Best Practices

1. Always use TLS in production - Set server.tls.enabled: true
2. Use strong API keys - Generate random, long keys
3. Enable fallback carefully - Only when you trust the fallback provider
4. Monitor logs - Watch for authentication failures
5. Regular updates - Keep dependencies up to date

Threat Model

• Protected: Man-in-the-middle attacks (via TLS), unauthorized access (via API keys)
• Not Protected: DDoS attacks (use a reverse proxy), rate limiting bypass (implement at reverse proxy level)

Monitoring

Metrics

Prometheus metrics available at :9090/metrics (server) and :9091/metrics (client).

Key metrics:

• securebridge_requests_total - Total requests
• securebridge_request_duration_seconds - Request latency
• securebridge_appliances_connected - Connected appliances
• securebridge_fallback_requests_total - Fallback requests

Health Checks

# Server health
curl http://localhost:8080/health

# Client health (local only)
curl http://localhost:9091/health

Troubleshooting

Client can't connect to server

• Check server URL in client config
• Verify API key matches server config
• Check firewall rules
• Enable debug logging: logging.level: debug

Requests timeout

• Increase request_timeout in server config
• Check local API is responding: curl http://localhost:11434/v1/models
• Monitor client logs for errors

Fallback not working

• Verify ANTHROPIC_API_KEY environment variable is set
• Check fallback.enabled: true in server config
• Review server logs for fallback triggers

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Areas for Contribution

• Additional fallback providers (OpenAI, Cohere, etc.)
• Client libraries in other languages
• Performance optimizations
• Documentation improvements

License

GNU AFFERO GENERAL PUBLIC LICENSE 3.0 - See LICENSE for details.

Support

• GitHub Issues: https://github.com/yourusername/securebridge/issues
• Documentation: See Document/TechnicalSpec.md

Roadmap

See Document/TechnicalSpec.md for detailed roadmap.

Version 1.0 (Current)

• Basic WebSocket bridge
• OpenAI-compatible API
• Single/multiple appliance routing
• Authentication & health checks

Future Versions

• Streaming responses (v1.1)
• Advanced monitoring UI (v1.2)
• Multi-tenant support (v2.0)