Skip to content

pmdnawaz1/synckey

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
examples
examples
 
 
src/synckey
src/synckey
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
pyproject.toml
pyproject.toml
 
 

Repository files navigation

synckey

Stars PyPI License
One API key. Every AI model. Zero headaches.

synckey lets your app use Gemini, Groq, OpenAI, Anthropic, Cerebras, Mistral, DeepSeek, xAI, and 40+ other providers — through a single OpenAI-compatible API key. No code changes. No rate limit nightmares. Just one endpoint.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8787/v1",
    api_key="sk-synckey-abc123..."      # ← one key for everything
)

# Just works. synckey routes to whichever provider has the model.
client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Write me a poem"}]
)

Why synckey?

Problem synckey Solution
Managing 10 different API keys One key for all providers
Rate limits breaking production Automatic failover to the next available provider
Expensive models during peak Tier-based fallback → uses cheaper models when others are throttled
Switching providers mid-code Zero code changes — just change the model name
Key rotation nightmares One command to rotate: synckey renew

Quick Start

# 1. Install
pip install synckey

# 2. Guided setup (30 seconds)
synckey setup

# 3. Start the gateway
synckey serve

Your app now uses http://localhost:8787/v1 with the unified key from setup.

That's it. Your OpenAI SDK, LangChain agent, or curl command now routes to 40+ models across all providers automatically.

How to Connect Your App

Python OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8787/v1",
    api_key="sk-synckey-xxx"
)

# Use default model (set during setup)
client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello!"}]
)

# Specify a model - synckey routes automatically
client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Streaming responses
stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)
for chunk in stream:
    print(chunk.choices[0].delta.content, end="")

Node.js

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:8787/v1',
  apiKey: 'sk-synckey-xxx'
});

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello!' }]
});

Go

package main

import (
    "github.com/sashabaranov/go-openai"
)

func main() {
    client := openai.NewClient("sk-synckey-xxx")
    client.BaseURL = "http://localhost:8787/v1"

    resp, err := client.CreateChatCompletion(
        context.Background(),
        openai.ChatCompletionRequest{
            Model: "gpt-4o",
            Messages: []openai.ChatMessage{
                {Role: "user", Content: "Hello!"},
            },
        },
    )
}

curl

# Basic request
curl http://localhost:8787/v1/chat/completions \
  -H "Authorization: Bearer sk-synckey-xxx" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

# Streaming
curl http://localhost:8787/v1/chat/completions \
  -H "Authorization: Bearer sk-synckey-xxx" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}], "stream": true}'

# Use default model (omit model)
curl http://localhost:8787/v1/chat/completions \
  -H "Authorization: Bearer sk-synckey-xxx" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello"}]}'

# Force specific provider
curl http://localhost:8787/v1/chat/completions \
  -H "Authorization: Bearer sk-synckey-xxx" \
  -H "X-Provider: groq" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-3.3-70b-versatile", "messages": [{"role": "user", "content": "Hello"}]}'

LangChain

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://localhost:8787/v1",
    api_key="sk-synckey-xxx",
    model="gpt-4o"
)

response = llm.invoke("Hello!")

API Reference

Endpoints

Endpoint Method Auth Description
/v1/chat/completions POST Yes Chat completions (streaming supported)
/v1/embeddings POST Yes Embeddings
/v1/completions POST Yes Legacy completions
/v1/models GET Yes List available models
/v1/usage GET Yes Token usage totals
/v1/events GET Yes Recent routing events
/v1/requests/{id} GET Yes Poll deferred request status
/healthz GET No Health check

Headers

Header Values Description
Authorization Bearer sk-synckey-xxx Required. Your unified API key
X-Quality-Floor frontier, high, mid, low Minimum tier to use (never drop below this)
X-Provider Provider ID (e.g., groq, gemini) Hint to prefer this provider first

POST /v1/chat/completions

Request:

{
  "model": "gpt-4o",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ],
  "temperature": 0.7,
  "max_tokens": 1000,
  "stream": false
}

Response (non-streaming):

{
  "id": "chatcmpl-xxx",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I help you today?"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 15,
    "total_tokens": 35
  }
}

Response (deferred/queued - 202):

{
  "id": "defer_abc123",
  "object": "chat.deferred",
  "status": "queued",
  "result_url": "/v1/requests/defer_abc123",
  "retry_after": 28
}

GET /v1/requests/{id}

Poll for deferred request completion:

curl http://localhost:8787/v1/requests/defer_abc123 \
  -H "Authorization: Bearer sk-synckey-xxx"

Queued response:

{"id": "defer_abc123", "status": "queued", "retry_after": 15}

Running response:

{"id": "defer_abc123", "status": "running"}

Done response:

{
  "id": "defer_abc123",
  "status": "done",
  "response": { /* full completion response */ }
}

Architecture

┌────────────────────────────────────────────────────────────────────────┐
│                           Your Application                             │
│                   (OpenAI SDK, LangChain, curl, any)                   │
└─────────────────────────────────────┬──────────────────────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────┐
                    │       synckey Gateway           │
                    │   http://localhost:8787/v1      │
                    │                                 │
                    │  ┌─────────────────────────┐    │
                    │  │  1. Auth Middleware     │    │
                    │  │     (unified key)       │    │
                    │  └─────────────────────────┘    │
                    │               │                 │
                    │               ▼                 │
                    │  ┌─────────────────────────┐    │
                    │  │  2. Router              │    │
                    │  │  - resolve model        │    │
                    │  │  - check tier floor     │    │
                    │  │  - priority ordering    │    │
                    │  └─────────────────────────┘    │
                    │               │                 │
                    │               ▼                 │
                    │  ┌─────────────────────────┐    │
                    │  │  3. Key Pool            │    │
                    │  │  - per-key buckets      │    │
                    │  │  - RPM/TPM tracking     │    │
                    │  │  - health states        │    │
                    │  └─────────────────────────┘    │
                    │               │                 │
                    │               ▼                 │
                    │  ┌─────────────────────────┐    │
                    │  │  4. Proxy Layer         │    │
                    │  │  - forward to upstream  │    │
                    │  │  - stream passthrough   │    │
                    │  │  - log usage/cost       │    │
                    │  └─────────────────────────┘    │
                    │               │                 │
                    │               ▼                 │
                    │  ┌─────────────────────────┐    │
                    │  │  5. Deferred Queue      │    │
                    │  │  - 202 + poll           │    │
                    │  │  - auto-retry           │    │
                    │  └─────────────────────────┘    │
                    └──────────────────────┬──────────┘
                                           │
               ┌───────────────────────────┼───────────────────────────┐
               ▼                           ▼                           ▼
          ┌─────────┐                ┌─────────┐                ┌─────────┐
          │  Groq   │                │ Gemini  │                │Anthropic│
          └─────────┘                └─────────┘                └─────────┘

Internal Components

Component Purpose Resources
Auth Middleware Validates unified key (SHA-256 hash) ~1MB RAM
Router Resolves model → provider, enforces tier floor ~5MB RAM
Key Pool Tracks per-key state (LIVE/COOLING/DEAD), RPM/TPM buckets ~10KB per key
Proxy Layer Forwards requests, handles streaming, logs responses Scales with concurrency
Deferred Queue SQLite-backed queue for when all keys are cooling ~1MB RAM + disk

Resource Usage

Metric Typical Under Load
RAM 50-100MB 200MB max
CPU <1% idle Scales with concurrent requests
Disk SQLite DB (~1MB per 10K requests) Configurable TTL
Latency Overhead 2-5ms Adds ~10ms at 100 concurrent

synckey is designed for minimal footprint. The gateway is single-threaded async by default (uvicorn with 1 worker). For higher throughput, increase workers:

synckey serve --workers 4   # 4 async workers

Smart Routing

synckey doesn't just blindly forward requests. It intelligently manages the entire lifecycle:

Automatic Failover

# You request: claude-opus-4-8
# Groq has it and is fast → routes there
# Groq hits rate limit → automatically tries Anthropic
# Anthropic succeeds → returns response, logs the fallback

Tier-Based Fallback

Every model is classified into a quality tier. If your preferred model is unavailable, synckey finds alternatives at the same tier first — never drops you to a cheaper tier unless explicitly configured.

FRONTIER: claude-opus-4-8, gpt-5, gemini-2.5-pro
HIGH:     claude-sonnet-4-6, gpt-4o, gemini-1.5-pro
MID:      gpt-4o-mini, llama-3.3-70b, gemini-2.0-flash
LOW:      gemma-2b, llama-3.2-1b

Deferred Requests (Queue)

When every key is rate-limited, synckey queues the request instead of failing:

{
  "id": "defer_abc123",
  "status": "queued",
  "result_url": "/v1/requests/defer_abc123",
  "retry_after": 28
}

Poll the result URL or wait — synckey runs it automatically when a key frees up. No code changes needed. Works with streaming too.

Use Cases

Load Balance Across Providers

# All these route intelligently based on your configured priority
client.chat.completions.create(model="gpt-4o")        # Groq → Gemini → OpenAI
client.chat.completions.create(model="claude-sonnet-4-6")  # Anthropic → Groq → Gemini
client.chat.completions.create(model="llama-3.3-70b")     # Groq → Cerebras → Together

Cost Optimization

# Set your default to a tier, not a specific model
synckey config set-default mid

# synckey picks the best MID-tier model that's available and affordable
# Falls back to HIGH tier only if MID is completely unavailable

Zero-Downtime Deployments

# When OpenAI has an outage, your app keeps working
# synckey automatically routes to the next available provider
# Your users never see an error

Live Dashboard

synckey dash
┌──────────────────────────────────────────────────────────────┐
│  synckey dashboard                             [1]overview   │
├──────────────────────────────────────────────────────────────┤
│                                                              │
│  Keys (3)        Requests Today          Events              │
│  ┌──────────┐    1,247                 12 rate limits        │
│  │ ● Groq   │    Avg: 142ms            2 tier fallbacks      │
│  │ ○ Gemini │    Errors: 0.2%           0 key deaths         │
│  │ ● Cohere │                                                │
│  └──────────┘    Last Request                                │
│                  Model: gpt-4o-mini                          │
│                  Provider: Groq                              │
│                  Latency: 89ms                               │
│                                                              │
│  [1]overview  [2]keys  [3]events  [4]routing  [q]quit        │
└──────────────────────────────────────────────────────────────┘

Supported Providers

40+ providers supported out of the box:

Category Providers
Fast/Cheap Groq, Cerebras, GitHub Models, DeepSeek
Frontier OpenAI, Anthropic, Google Gemini, xAI
Open Source Ollama, LM Studio, LocalAI, Fireworks
Aggregators OpenRouter, Together AI, SambaNova

Run synckey providers to see the full list.

CLI Reference

# First time setup
synckey setup                      # guided interactive setup
synckey setup --force              # regenerate unified key

# Gateway
synckey serve                      # start (runs in background)
synckey serve --port 9000        # custom port

# Key management
synckey keys                       # list all keys
synckey keys --rm <id>            # remove a key
synckey keys --disable <id>       # temporarily disable
synckey keys --enable <id>       # re-enable a disabled key

# Testing
synckey test                       # health check all keys
synckey test groq llama-3.3-70b-versatile 5  # test a route

# Routing
synckey routing priority              # show provider order
synckey routing priority groq gemini  # set preference
synckey routing preset reliable       # never drop tier
synckey routing preset cheap          # prefer cost over quality

# Config
synckey config set-default gemini-2.0-flash    # set default model
synckey config provider-default groq llama-3.3-70b-versatile  # per-provider
synckey config alias fast mid         # create alias

# Tiers
synckey tier set my-model frontier    # override tier
synckey tier list                    # show all overrides

# Monitoring
synckey dash                        # live dashboard
synckey status                      # quick overview
synckey usage                       # totals
synckey usage --spend              # cost breakdown
synckey usage --recent              # last 25 requests
synckey events                      # rate limits, fallbacks, deaths
synckey deferred list               # queued requests

# Key rotation
synckey renew                       # regenerate unified key

Environment Variables

Variable Default Description
SYNCKEY_HOME ~/.synckey/ Config and state directory
SYNCKEY_DB {SYNCKEY_HOME}/synckey.db SQLite database path
Provider env vars — GROQ_API_KEY, ANTHROPIC_API_KEY, etc.

Configuration

Config file: ~/.synckey/config.toml

[gateway]
host = "127.0.0.1"
port = 8787
request_timeout = 120

[routing]
priority = ["groq", "gemini"]
tier_fallback = true
max_retries = 6
default_cooldown = 20
deep_cooling_threshold = 60

[deferred]
enabled = true
ttl = 3600
poll = 5
max_queue = 1000

Production Deployment

PM2 (Recommended for all platforms)

PM2 keeps synckey running 24/7, auto-restarts on crash, and survives server reboots.

# Install PM2
npm install -g pm2

# Start synckey (runs in background, daemonized)
pm2 start examples/ecosystem.config.js

# Save current process list (persists across reboots)
pm2 save

# Generate startup script for your OS
pm2 startup

# Other useful commands
pm2 logs synckey        # view logs
pm2 restart synckey       # restart
pm2 stop synckey          # stop
pm2 monit                 # real-time dashboard

ecosystem.config.js (included in examples/):

module.exports = {
  apps: [{
    name: 'synckey',
    script: 'python.exe',
    args: '-m synckey serve',
    cwd: 'C:\\path\\to\\synckey',
    interpreter: 'none',
    watch: false,
    autorestart: true,
    restart_delay: 10000,
    max_restarts: 10,
    min_uptime: 5000,
  }]
};

systemd (Linux/WSL)

For servers running systemd:

# 1. Install synckey
pip install synckey

# 2. Create dedicated user (optional but recommended)
sudo useradd -r -s /bin/false synckey
sudo mkdir /var/lib/synckey /run/synckey
sudo chown synckey:synckey /var/lib/synckey /run/synckey

# 3. Install service file
sudo cp examples/synckey.service /etc/systemd/system/
sudo systemctl daemon-reload

# 4. Enable and start
sudo systemctl enable synckey
sudo systemctl start synckey
sudo systemctl status synckey

synckey.service (included in examples/):

[Unit]
Description=synckey unified AI gateway
Documentation=https://github.com/pmdnawaz1/synckey
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=synckey
Group=synckey
WorkingDirectory=/opt/synckey
ExecStart=/usr/bin/python3 -m synckey serve
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=synckey

# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/synckey /run/synckey
Environment=PYTHONUNBUFFERED=1

[Install]
WantedBy=multi-user.target

Windows Service (NSSM)

nssm install synckey "C:\path\to\python.exe" "-m synckey serve"
nssm set synckey AppDirectory "C:\path\to\synckey"
nssm set synckey Start SERVICE_AUTO_START
nssm start synckey

Auto-Restart Behavior

Event Action
Server reboot PM2/systemd auto-starts synckey
Crash (OOM, segfault) Auto-restart after 10s delay
All keys cooling synckey returns 202 + poll URL (no crash)

Expose on Domain

synckey serve --host 0.0.0.0 --port 8787

nginx:

server {
    listen 443 ssl;
    server_name ai.example.com;
    ssl_certificate /etc/ssl/certs/ai.example.com.crt;
    ssl_certificate_key /etc/ssl/private/ai.example.com.key;

    location / {
        proxy_pass http://127.0.0.1:8787;
        proxy_set_header Host $host;
        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}

Caddy:

ai.example.com {
    reverse_proxy localhost:8787 {
        flush_interval -1
    }
}

Security

  • Provider secrets are Fernet-encrypted at rest
  • Unified key stored as SHA-256 hash (not reversible)
  • Gateway binds to localhost by default
  • Secrets directory is 0700 permissions

Troubleshooting

401 Unauthorized

  • Verify your unified key is correct: synckey status
  • Regenerate if needed: synckey renew

Connection refused

  • Ensure synckey is running: synckey serve
  • Check if port is in use: netstat -an | grep 8787

All requests returning 202 queued

  • Keys may be in cooldown: synckey status
  • Check events: synckey events

Model not found (404)

  • Model may not be in index: synckey models --refresh
  • Provider may not have the model

Compare

Feature synckey LiteLLM Portkey Direct
One API key ✅ ✅ ✅ ❌
Auto-failover ✅ ✅ ✅ ❌
Tier fallback ✅ ❌ ❌ ❌
Deferred queue ✅ ❌ ❌ ❌
No DB needed ✅ ❌ ❌ ✅
Local gateway ✅ ❌ ❌ ✅
Zero config ✅ ❌ ❌ ✅

License

MIT

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages