Skip to content

goweraa/api-homelab

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
grafana
grafana
 
 
kong
kong
 
 
loki
loki
 
 
postgres
postgres
 
 
prometheus
prometheus
 
 
promtail
promtail
 
 
services
services
 
 
.env.example
.env.example
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
docker-compose.yml
docker-compose.yml
 
 

Repository files navigation

API Homelab

A self-contained API platform running on Docker Compose that demonstrates production-quality patterns: API gateway management, multi-service orchestration, and a full observability stack (metrics, logs, dashboards).

Table of Contents

Architecture

                      ┌──────────────────────────────────────────────┐
                      │              Kong API Gateway :8080           │
  Client  ──:8080──►  │  key-auth · rate-limit · correlation-id      │
                      │  prometheus plugin → metrics on :8100        │
                      └──────┬────────────┬────────────┬─────────────┘
                             │            │            │
                   /api/v1/users  /api/v1/products  /api/v1/weather
                             │            │            │
                       ┌─────▼──┐   ┌─────▼──┐   ┌───▼──────┐
                       │ users  │   │products│   │ weather  │
                       │  api   │   │  api   │   │  mock    │
                       └─────┬──┘   └─────┬──┘   └──────────┘
                             │            │         (stateless)
                       ┌─────▼────────────▼──┐
                       │      PostgreSQL      │
                       │  tables: users,      │
                       │          products    │
                       └─────────────────────┘

  ┌─────────────────────────────────────────────────────┐
  │                   Observability                     │
  │                                                     │
  │  Prometheus ← scrapes Kong (:8100) + APIs (/metrics)│
  │  Promtail (Docker socket) → Loki → Grafana :3000    │
  │  Redis ← Kong rate-limit counter store              │
  └─────────────────────────────────────────────────────┘

Request flow: Every client request enters through Kong on port 8080. Kong enforces API key authentication, applies rate limits (60 req/min per consumer, backed by Redis), injects a X-Request-ID correlation header, and routes the request to the correct upstream service. The upstream services never see the API key — Kong strips it before forwarding.

Stack

Component Image Purpose
Kong kong:3.7.1 API gateway — auth, rate-limiting, routing, metrics
Redis redis:7.2-alpine Kong rate-limit counter store (ephemeral, no persistence)
users-api local build (Python 3.12) FastAPI — full CRUD user management with PostgreSQL
products-api local build (Python 3.12) FastAPI — product catalogue with stock management
weather-mock local build (Python 3.12) FastAPI — mock weather API with chaos engineering endpoints
PostgreSQL postgres:16.3-alpine Persistent storage; tables created by services on startup
Prometheus prom/prometheus:v2.53.0 Metrics collection, 7-day retention
Loki grafana/loki:3.1.0 Log aggregation (filesystem storage, tsdb schema v13)
Promtail grafana/promtail:3.1.0 Reads Docker container logs → ships to Loki
Grafana grafana/grafana:11.1.0 Dashboards; auto-provisioned on startup

Prerequisites

  • Docker Engine ≥ 24 with Docker Compose v2 (docker compose — note: no hyphen)
  • Ports 8080, 3000, 9090 available on the host
  • gh CLI — only needed if pushing to GitHub

Quick Start

# 1. Clone the repository
git clone https://github.com/goweraa/api-homelab.git
cd api-homelab

# 2. Create your .env file and set passwords
cp .env.example .env
# Edit .env — change POSTGRES_PASSWORD and GF_SECURITY_ADMIN_PASSWORD at minimum

# 3. Build the three FastAPI images and start all 10 services
docker compose up -d --build

# 4. Wait ~30 seconds for health checks to pass
docker compose ps    # all services should show "healthy"

# 5. Verify the gateway is up
curl http://localhost:8080/api/v1/users/health
# → {"status":"healthy","service":"users-api","version":"1.0.0","database":"healthy"}

# 6. Make your first authenticated API call
curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/users/

Open http://localhost:3000 for the Grafana dashboard (no login needed for read-only view).

Environment Variables

Copy .env.example to .env and fill in real values. The file is gitignored and never committed.

Variable Description Default in .env.example
POSTGRES_USER PostgreSQL username apiuser
POSTGRES_PASSWORD PostgreSQL password changeme_strong_password
POSTGRES_DB Database name apidb
USERS_DB_URL Full DB connection URL for users-api assembled from above
PRODUCTS_DB_URL Full DB connection URL for products-api assembled from above
KONG_DEMO_API_KEY Demo consumer API key (for reference) demo-api-key-abc123
KONG_INTERNAL_API_KEY Internal service API key (for reference) internal-api-key-xyz789
GF_SECURITY_ADMIN_USER Grafana admin username admin
GF_SECURITY_ADMIN_PASSWORD Grafana admin password changeme_grafana_password

Note: API keys are defined in kong/kong.yml under consumers. The .env values are for reference only — to rotate a key, update kong/kong.yml and run docker exec api-kong kong reload.

Accessing Services

Service URL Auth required
API Gateway (all APIs) http://localhost:8080 Yes — x-api-key header
Grafana dashboards http://localhost:3000 No (anonymous viewer enabled)
Prometheus UI http://localhost:9090 No
Users API — Swagger UI http://localhost:8080/api/v1/users/docs Yes — x-api-key header
Products API — Swagger UI http://localhost:8080/api/v1/products/docs Yes — x-api-key header
Weather API — Swagger UI http://localhost:8080/api/v1/weather/docs Yes — x-api-key header

API keys (from .env.example, safe to use in dev):

Consumer Key Use
demo-client demo-api-key-abc123 General testing
internal-service internal-api-key-xyz789 Simulated service-to-service calls

Seed Data

Both database-backed services auto-create their tables and seed initial data on first startup (only if the table is empty).

Users (5 records):

ID Name Email Role Department
1 Alice Admin alice@example.com admin Engineering
2 Bob Editor bob@example.com editor Marketing
3 Carol Viewer carol@example.com viewer Sales
4 Dan Dev dan@example.com editor Engineering
5 Eve Analyst eve@example.com viewer Data

Products (8 records — homelab-themed):

ID Name SKU Category Price Stock
1 Raspberry Pi 5 (8GB) RPI5-8GB SBC $80.00 25
2 NVMe SSD 1TB SSD-NVME-1TB Storage $79.99 50
3 10GbE Network Switch 8-port NET-10GBE-8P Networking $299.00 10
4 32GB DDR5 RAM Kit RAM-DDR5-32G Memory $119.99 30
5 Proxmox VE Support Subscription PVE-SUB-1Y Software $149.00 999
6 Structured Cabling Kit CAB-CAT6-KIT Networking $45.00 15
7 Mini-ITX Server Case CASE-MITX-1U Hardware $89.99 8
8 UPS 1500VA UPS-1500VA Power $189.00 5

API Reference

All requests must go through Kong on port 8080 with the header x-api-key: demo-api-key-abc123.

Users API

Base path: /api/v1/users

Full interactive docs: http://localhost:8080/api/v1/users/docs

GET /api/v1/users

Returns a paginated list of users. Supports filtering by role and department.

Query parameters:

Parameter Type Default Description
page int 1 Page number (1-indexed)
page_size int 20 Results per page (max 100)
role string — Filter by role: admin, editor, or viewer
department string — Case-insensitive partial match on department name 
# All users
curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/users/

# Only admins
curl -H "x-api-key: demo-api-key-abc123" "http://localhost:8080/api/v1/users/?role=admin"

# Users in Engineering (partial match)
curl -H "x-api-key: demo-api-key-abc123" "http://localhost:8080/api/v1/users/?department=eng"

# Page 2, 2 results per page
curl -H "x-api-key: demo-api-key-abc123" "http://localhost:8080/api/v1/users/?page=2&page_size=2"

Response:

{
  "total": 5,
  "page": 1,
  "page_size": 20,
  "pages": 1,
  "items": [{ "id": 1, "name": "Alice Admin", "email": "alice@example.com", "role": "admin", ... }]
}

POST /api/v1/users

Creates a new user. Returns 201 Created on success, 409 Conflict if the email is already registered.

Required fields: name, email Optional fields: role (default: viewer), department, bio Role must be one of: admin, editor, viewer

curl -X POST http://localhost:8080/api/v1/users/ \
  -H "x-api-key: demo-api-key-abc123" \
  -H "Content-Type: application/json" \
  -d '{"name": "Frank Engineer", "email": "frank@example.com", "role": "editor", "department": "Engineering"}'

GET /api/v1/users/{user_id}

Returns a single user by ID. Returns 404 Not Found if the ID does not exist.

curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/users/1

PUT /api/v1/users/{user_id}

Partially updates a user — only fields present in the request body are changed. Email cannot be changed after creation.

# Promote a user to admin and update their department
curl -X PUT http://localhost:8080/api/v1/users/3 \
  -H "x-api-key: demo-api-key-abc123" \
  -H "Content-Type: application/json" \
  -d '{"role": "admin", "department": "Platform"}'

DELETE /api/v1/users/{user_id}

Deletes a user permanently. Returns 204 No Content on success.

curl -X DELETE -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/users/5

GET /api/v1/users/search/by-email

Exact email lookup. Useful for checking if an email is already registered.

curl -H "x-api-key: demo-api-key-abc123" \
  "http://localhost:8080/api/v1/users/search/by-email?email=alice@example.com"

Products API

Base path: /api/v1/products

Full interactive docs: http://localhost:8080/api/v1/products/docs

GET /api/v1/products

Returns a paginated, filterable list of products.

Query parameters:

Parameter Type Default Description
page int 1 Page number
page_size int 20 Results per page (max 100)
category string — Exact category match (e.g. Networking)
q string — Full-text search on name and description
price_min decimal — Minimum price filter
price_max decimal — Maximum price filter
in_stock bool — If true, only products with stock > 0
active_only bool true Exclude discontinued products 
# All products
curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/products/

# Networking products under $100
curl -H "x-api-key: demo-api-key-abc123" \
  "http://localhost:8080/api/v1/products/?category=Networking&price_max=100"

# Search for "SSD" in name or description, in stock only
curl -H "x-api-key: demo-api-key-abc123" \
  "http://localhost:8080/api/v1/products/?q=SSD&in_stock=true"

POST /api/v1/products

Creates a product. SKUs are automatically uppercased and trimmed. Returns 409 Conflict if the SKU already exists.

curl -X POST http://localhost:8080/api/v1/products/ \
  -H "x-api-key: demo-api-key-abc123" \
  -H "Content-Type: application/json" \
  -d '{"name": "USB-C Hub 7-port", "sku": "USB-HUB-7P", "category": "Hardware", "price": 39.99, "stock": 20}'

GET /api/v1/products/{product_id}

Returns a single product by ID.

PUT /api/v1/products/{product_id}

Partially updates a product. SKU cannot be changed after creation.

# Mark a product as discontinued
curl -X PUT http://localhost:8080/api/v1/products/7 \
  -H "x-api-key: demo-api-key-abc123" \
  -H "Content-Type: application/json" \
  -d '{"is_active": false}'

PATCH /api/v1/products/{product_id}/stock

Adjusts stock by a delta value. Positive delta adds stock, negative removes it. Returns 422 Unprocessable Entity if the adjustment would make stock go below zero.

# Receive a shipment (+10 units)
curl -X PATCH http://localhost:8080/api/v1/products/1/stock \
  -H "x-api-key: demo-api-key-abc123" \
  -H "Content-Type: application/json" \
  -d '{"delta": 10, "reason": "Received new shipment"}'

# Sell 3 units
curl -X PATCH http://localhost:8080/api/v1/products/1/stock \
  -H "x-api-key: demo-api-key-abc123" \
  -H "Content-Type: application/json" \
  -d '{"delta": -3, "reason": "Customer order #1042"}'

DELETE /api/v1/products/{product_id}

Permanently deletes a product. Returns 204 No Content.

GET /api/v1/products/categories/list

Returns all distinct category names currently in the catalogue.

curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/products/categories/list
# → ["Hardware", "Memory", "Networking", "Power", "SBC", "Software", "Storage"]

Weather Mock API

Base path: /api/v1/weather

Full interactive docs: http://localhost:8080/api/v1/weather/docs

This service simulates a third-party weather data provider. It is stateless — no database. It also hosts the chaos engineering endpoints described in the next section.

Supported cities: london, new-york, tokyo, sydney, cape-town, kigali, berlin, dubai

GET /api/v1/weather/cities/supported

Lists all supported city slugs and their metadata.

curl -H "x-api-key: demo-api-key-abc123" \
  http://localhost:8080/api/v1/weather/cities/supported

GET /api/v1/weather/{city}

Returns current weather conditions for a city. Data is pseudo-random but seeded deterministically from the city name and current hour, so it changes hourly.

curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/weather/london
curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/weather/kigali

Response fields: city, country, coordinates, timezone, timestamp, weather (condition, temperature_c, feels_like_c, humidity_pct, wind_speed_kmh, wind_direction, visibility_km, uv_index, pressure_hpa), cache_ttl_seconds

GET /api/v1/weather/{city}/forecast

Returns a multi-day forecast. Days defaults to 5, maximum is 14.

# 7-day forecast for Tokyo
curl -H "x-api-key: demo-api-key-abc123" \
  "http://localhost:8080/api/v1/weather/tokyo/forecast?days=7"

Chaos Engineering Endpoints

These endpoints live under /api/v1/weather/chaos/ and are designed to create observable failure conditions. Use them to generate interesting data in Grafana, practice troubleshooting, or demonstrate resilience concepts.

GET /api/v1/weather/chaos/slow — simulate upstream latency

Sleeps for delay seconds before responding.

Parameter Default Range Notes
delay 5.0 0.1 – 30.0 Seconds to wait 
# Cause a 504 Gateway Timeout from Kong (Kong's read_timeout is 15s)
curl -H "x-api-key: demo-api-key-abc123" \
  "http://localhost:8080/api/v1/weather/chaos/slow?delay=16"

What to watch in Grafana: The Kong latency panel will show a spike. At delay ≥ 15s, Kong returns a 504 and you'll see it on the 5xx panel.

GET /api/v1/weather/chaos/error — return a specific HTTP error

Forces the endpoint to return any 4xx or 5xx status code.

Parameter Default Description
code 500 HTTP status code (400–599)
message "Simulated upstream error" Error detail text 
# Simulate a 503 Service Unavailable
curl -H "x-api-key: demo-api-key-abc123" \
  "http://localhost:8080/api/v1/weather/chaos/error?code=503"

Demo: Hit this endpoint 10–20 times and watch the error rate stat panel in Grafana spike from 0% to a high value.

GET /api/v1/weather/chaos/flaky — intermittent random failures

Randomly fails a configurable proportion of requests.

Parameter Default Range Description
fail_rate 0.5 0.0 – 1.0 Probability of failure per request
error_code 503 4xx/5xx HTTP status code when failing 
# Loop 50 requests — expect ~50% errors
for i in $(seq 50); do
  curl -s -o /dev/null -w "%{http_code}\n" \
    -H "x-api-key: demo-api-key-abc123" \
    "http://localhost:8080/api/v1/weather/chaos/flaky?fail_rate=0.5"
done

What to watch in Grafana: The per-service error rate panel will show ~50% errors on the weather-mock job. This demonstrates what intermittent upstream failures look like in a real observability stack.

POST /api/v1/weather/chaos/toggle — global circuit-breaker simulation

Toggles global chaos mode ON or OFF. When ON, all weather endpoints return 503 Service Unavailable. Call again to restore normal operation.

# Enable chaos mode — all weather requests will fail with 503
curl -X POST -H "x-api-key: demo-api-key-abc123" \
  http://localhost:8080/api/v1/weather/chaos/toggle

# Confirm it's on — should return 503
curl -H "x-api-key: demo-api-key-abc123" http://localhost:8080/api/v1/weather/london

# Disable chaos mode — restore normal operation
curl -X POST -H "x-api-key: demo-api-key-abc123" \
  http://localhost:8080/api/v1/weather/chaos/toggle

Demo talking point: Use this to simulate a full upstream provider outage. Discuss retry strategies, circuit breakers, fallback caching, and SLA considerations.

Kong Gateway

Kong runs in DB-less declarative mode — all configuration lives in kong/kong.yml. There is no database backing Kong itself.

Reload config without restarting

docker exec api-kong kong reload

Active plugins (applied globally)

Plugin Configuration Effect
key-auth Header: x-api-key; hide_credentials: true Requires API key on every request; strips the key before forwarding upstream so it never appears in service logs
rate-limiting 60 req/min, 1000 req/hr per consumer; Redis-backed; fault_tolerant: true Tracks per-consumer counters in Redis. If Redis is unavailable, requests pass through (fail-open). Response headers include X-RateLimit-Remaining-Minute
prometheus status code, latency, bandwidth, upstream health metrics Exposes metrics at http://kong:8100/metrics, scraped by Prometheus
correlation-id Header: X-Request-ID; generator: UUID; echo_downstream: true Every request and response carries a unique ID — essential for correlating Kong logs with upstream service logs in Loki

Consumers

Username API Key Rate limit
demo-client demo-api-key-abc123 60 req/min
internal-service internal-api-key-xyz789 60 req/min

Routes

Route Path Methods
users-route /api/v1/users GET, POST, PUT, PATCH, DELETE
products-route /api/v1/products GET, POST, PUT, PATCH, DELETE
weather-route /api/v1/weather GET, POST

All routes use strip_path: false, which means the full /api/v1/... path is preserved and forwarded to the upstream service as-is.

Observability

Grafana Dashboards

Open http://localhost:3000. The API Homelab Overview dashboard auto-provisions on startup with three sections:

Kong Gateway

  • Total request rate (req/s)
  • Error rate (4xx + 5xx combined)
  • p99 latency stat panel
  • Kong service uptime
  • Request rate by status class (2xx / 4xx / 5xx time series)
  • Latency percentiles (p50 / p95 / p99 time series)

Per-Service Health

  • Uptime status panels for all three services
  • Request rate per service
  • p95 latency per service

Live Logs

  • All service logs (filterable)
  • Error-only log panel (regex filter for error|exception|traceback|critical)
  • Kong access logs

The dashboard refreshes every 30 seconds. To edit it, log in with the admin credentials from .env.

Prometheus

Scrapes every 15 seconds from:

Target Address Key metrics
Kong kong:8100/metrics kong_http_requests_total, kong_request_latency_ms_bucket, kong_upstream_target_health
users-api users-api:8000/metrics http_requests_total, http_request_duration_seconds_bucket
products-api products-api:8000/metrics http_requests_total, http_request_duration_seconds_bucket
weather-mock weather-mock:8000/metrics http_requests_total, http_request_duration_seconds_bucket

Metrics are scraped directly from services, bypassing Kong.

# Reload Prometheus config without restarting the container
curl -X POST http://localhost:9090/-/reload

Loki + Promtail

Promtail reads container logs via the Docker socket (/var/run/docker.sock) using docker_sd_configs. It only scrapes containers in the api-homelab compose project (filtered by the com.docker.compose.project label).

Each log entry is labelled with compose_service and compose_project, making it easy to query in Grafana.

Useful LogQL queries (paste into Grafana → Explore → Loki):

# All logs from this project
{compose_project="api-homelab"}

# Errors only, across all services
{compose_project="api-homelab"} |~ `(?i)(error|exception|traceback|critical)`

# Specific service logs
{compose_service="users-api"}
{compose_service="products-api"}
{compose_service="weather-mock"}
{compose_service="kong"}

# Filter for a specific request by correlation ID
{compose_project="api-homelab"} |= `<paste X-Request-ID value here>`

# Watch chaos events in real time
{compose_service="weather-mock"} |~ `chaos=`

Useful Commands

# Start all services in the background
docker compose up -d --build

# Check health status of all containers
docker compose ps

# Follow logs for a specific service
docker compose logs -f users-api
docker compose logs -f kong

# Rebuild and restart a single service after code changes
docker compose up -d --build users-api

# Reload Kong config after editing kong/kong.yml
docker exec api-kong kong reload

# Reload Prometheus config after editing prometheus/prometheus.yml
curl -X POST http://localhost:9090/-/reload

# Open a psql shell on the database
docker exec -it api-postgres psql -U apiuser -d apidb

# Stop all services (preserves volumes)
docker compose down

# Destroy everything including all data volumes (clean slate)
docker compose down -v

Security Notes

  • .env is gitignored — never commit real secrets; only .env.example is tracked
  • Kong strips API keys (hide_credentials: true) before forwarding requests — keys never appear in upstream service logs
  • Kong admin API is bound to 127.0.0.1:8001 only and not port-mapped to the host
  • All FastAPI containers run as a non-root appuser (created in the Dockerfile)
  • Grafana anonymous access is enabled in grafana/grafana.ini for portfolio viewing — set auth.anonymous.enabled = false to require login in production
  • Redis has no password — acceptable for a local homelab; add requirepass to the Redis command in docker-compose.yml for production use

Folder Structure

api-homelab/
├── docker-compose.yml              # Orchestrates all 10 services
├── .env.example                    # All required env vars (no real values)
├── .gitignore
├── README.md
│
├── kong/
│   └── kong.yml                    # DB-less declarative gateway config
│
├── services/
│   ├── users-api/
│   │   ├── Dockerfile
│   │   ├── main.py                 # FastAPI app — users CRUD
│   │   └── requirements.txt
│   ├── products-api/
│   │   ├── Dockerfile
│   │   ├── main.py                 # FastAPI app — products + stock management
│   │   └── requirements.txt
│   └── weather-mock/
│       ├── Dockerfile
│       ├── main.py                 # FastAPI app — weather mock + chaos endpoints
│       └── requirements.txt
│
├── postgres/
│   └── init.sql                    # Runs once on first container creation
│
├── prometheus/
│   └── prometheus.yml              # Scrape targets config
│
├── loki/
│   └── loki-config.yml             # Single-binary mode, filesystem storage
│
├── promtail/
│   └── promtail-config.yml         # Docker socket log scraping config
│
└── grafana/
    ├── grafana.ini                 # Anonymous access, port, log level
    └── provisioning/
        ├── datasources/
        │   └── datasources.yml     # Auto-configures Prometheus + Loki
        └── dashboards/
            ├── dashboards.yml      # Dashboard provider config
            └── api-overview.json   # Main dashboard (auto-loads on startup)

About

Kong API gateway + FastAPI microservices + Prometheus/Loki/Grafana observability stack on Docker Compose

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages