Skip to content

vadimsemenykv/saboteur

Repository files navigation

Saboteur — HTTP Fault Injection Proxy

A lightweight HTTP-layer fault injection proxy. Intercepts traffic and injects configurable faults based on rules — useful for testing service resilience in development and CI/CD.

Key differentiator from Toxiproxy: operates at HTTP layer — understands URLs, methods, headers, and response bodies. Not TCP-level.


Quickstart

docker run -p 8080:8080 -p 8081:8081 \
  -e UPSTREAM_URL=http://my-service:3000 \
  yourname/saboteur
  • Port 8080 — proxy (send your real traffic here)
  • Port 8081 — control UI + API (manage rules, view traffic)

Open http://localhost:8081 in your browser to use the web UI.


With a config file

docker run -p 8080:8080 -p 8081:8081 \
  -v $(pwd)/config/example.yaml:/etc/saboteur/config.yaml \
  yourname/saboteur --config /etc/saboteur/config.yaml

Docker Compose

docker compose up

Uses docker-compose.yml in the repo root. Spins up saboteur + a mock upstream (hashicorp/http-echo).


Configuration Reference

All settings can be provided via YAML file or environment variables.

YAML key Env var Default Description
proxy.upstream_url UPSTREAM_URL (required) URL to proxy traffic to
proxy.port PROXY_PORT 8080 Proxy listener port
proxy.timeout_ms 30000 Upstream request timeout
proxy.max_idle_conns 100 Max idle upstream connections
control.port CONTROL_PORT 8081 Control API + UI port
control.bind CONTROL_BIND 0.0.0.0 Control bind address
control.api_key API_KEY (none) If set, require X-API-Key header on control API
traffic_log.max_entries TRAFFIC_LOG_SIZE 1000 Ring-buffer size
log_level LOG_LEVEL info debug / info / warn / error
log_format LOG_FORMAT json json / text

Fault Types

Latency

Add delay before forwarding or returning.

fault:
  type: latency
  mode: fixed       # fixed | uniform | normal
  fixed_ms: 500
  apply_to: request # request | response

Error Response

Return a specific status without hitting upstream.

fault:
  type: error
  status_code: 503
  body: '{"error":"down"}'
  headers:
    Retry-After: "30"

Connection Abort

Drop the connection after N bytes.

fault:
  type: abort
  after_bytes: 0    # 0 = drop immediately

Timeout

Accept connection, forward to upstream, never respond.

fault:
  type: timeout

Body Corruption

Replace the upstream response body.

fault:
  type: body_corrupt
  mode: json_invalid  # empty | random_bytes | json_invalid | truncate
  truncate_bytes: 100 # for truncate mode

Header Injection

Add or override headers on request or response.

fault:
  type: header_inject
  apply_to: response   # request | response
  headers:
    X-Injected: "true"

Bandwidth Throttle

Rate-limit response streaming.

fault:
  type: throttle
  bytes_per_second: 1024

Rule System

Rules are evaluated in priority order (lower number = higher priority). A request matches the first rule where all matcher conditions are satisfied.

rules:
  - id: "payment-errors"
    enabled: true
    priority: 10
    description: "503 on 20% of EU payment POSTs"
    match:
      path_prefix: "/api/payment"
      methods: [POST]
      headers:
        X-Region: "EU"
    percentage: 20        # 0-100; fraction of matching requests that get the fault
    fault:
      type: error
      status_code: 503
      body: '{"error":"unavailable"}'

Matcher fields (all optional; unset = match anything)

Field Description
path Exact URL path
path_prefix URL path prefix
path_regex RE2 regex on URL path
methods HTTP methods (empty = all)
headers All listed headers must match
query_params All listed query params must match

Control API Overview

Base URL: http://localhost:8081

Method Path Description
GET /health Health check
GET /metrics Prometheus metrics
GET /api/rules List all rules
POST /api/rules Create rule
GET /api/rules/{id} Get rule
PUT /api/rules/{id} Replace rule
PATCH /api/rules/{id} Update fields (enabled, priority, percentage)
DELETE /api/rules/{id} Delete rule
POST /api/rules/reset Delete all runtime rules (preserves config-file rules)
GET /api/traffic Traffic log (?limit=100&path_filter=&fault_only=true)
DELETE /api/traffic Clear traffic log
GET /api/traffic/stream SSE stream of live traffic
GET /api/config Current effective configuration

Full spec: openapi.yaml

API Key auth

curl -H "X-API-Key: mysecret" http://localhost:8081/api/rules

Quick examples

# Add an error rule
curl -X POST http://localhost:8081/api/rules \
  -H "Content-Type: application/json" \
  -d '{"id":"test-error","enabled":true,"priority":1,"percentage":100,"match":{},"fault":{"type":"error","status_code":503}}'

# Toggle a rule off
curl -X PATCH http://localhost:8081/api/rules/test-error \
  -H "Content-Type: application/json" \
  -d '{"enabled":false}'

# View recent traffic (faults only)
curl "http://localhost:8081/api/traffic?limit=10&fault_only=true"

# Delete a rule
curl -X DELETE http://localhost:8081/api/rules/test-error

# Reset all runtime rules
curl -X POST http://localhost:8081/api/rules/reset

Hot Reload (SIGHUP)

Send SIGHUP to reload the config file without restarting:

kill -HUP $(pgrep saboteur)

Config-file rules are replaced atomically. Runtime rules (created via API) are preserved.


Prometheus Metrics

Available at http://localhost:8081/metrics.

Metric Labels Description
fault_proxy_requests_total method, path_pattern, status_code, fault_type Total proxied requests
fault_proxy_faults_total rule_id, fault_type Requests where a fault was injected
fault_proxy_request_duration_ms quantile End-to-end latency (includes injected latency)
fault_proxy_upstream_duration_ms quantile Upstream-only latency
fault_proxy_rules_active Number of enabled rules
fault_proxy_upstream_healthy 1 = reachable, 0 = unreachable

Building from source

go build -o saboteur ./cmd/saboteur
UPSTREAM_URL=http://localhost:3000 ./saboteur

Run tests

go test ./...
go test -race ./...

Build Docker image

docker build -f docker/Dockerfile -t saboteur .

Multi-arch:

docker buildx build --platform linux/amd64,linux/arm64 \
  -f docker/Dockerfile -t yourname/saboteur:latest --push .

Out of scope

  • TCP-level fault injection (use Toxiproxy)
  • gRPC support
  • Persistent rule storage (config file is the persistence layer)
  • TLS termination on the proxy port

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors