Hub-and-Spoke Architecture with Docker Compose

A complete, production-ready implementation of a hub-and-spoke distributed architecture using Docker Compose. This architecture separates the central coordination layer (Hub) from the execution services (Spokes), enabling scalability, flexibility, and fault isolation.

Architecture Overview

Hub Components (Central Coordination)

The Hub acts as the central nervous system, coordinating all communication and orchestration:

• Hub Gateway (NGINX): Entry point for all external traffic, load balancing, and SSL termination
• Redis Broker: High-speed message broker for real-time communication and caching
• RabbitMQ Broker: Message queue for asynchronous task distribution and event streaming
• Hub Orchestrator: Central brain that routes requests, manages state, and coordinates spoke services

Spoke Components (Execution Services)

The Spokes are independent services that perform specific tasks:

1. Authentication Service (Node.js): User authentication, JWT token management
2. User Service (Node.js): User profile management, CRUD operations
3. Notification Service (Node.js): Email, SMS, and push notification delivery
4. Analytics Service (Python): Data processing, metrics aggregation, reporting
5. Storage Service (MinIO): Object storage for files, images, and documents
6. Search Service (Elasticsearch): Full-text search and indexing

Supporting Infrastructure

• PostgreSQL: Relational database for structured data
• Prometheus: Metrics collection and monitoring
• Grafana: Visualization dashboards and alerting

Key Features

Hub-and-Spoke Benefits

✅ Centralized Control: All routing and orchestration logic in one place
✅ Independent Scaling: Scale spokes independently based on load
✅ Fault Isolation: Spoke failures don't cascade to other services
✅ Flexible Communication: Synchronous (HTTP) and asynchronous (message queue) patterns
✅ Easy Monitoring: Centralized metrics and logging
✅ Service Discovery: Hub maintains registry of all active spokes

Architecture Characteristics

• Separation of Concerns: Hub (brain) vs. Spokes (hands)
• Loose Coupling: Services communicate only through the hub
• High Availability: Health checks and automatic restarts
• Security: Network isolation, environment-based secrets
• Observability: Prometheus metrics + Grafana dashboards

Prerequisites

• Docker Engine 20.10+
• Docker Compose 2.0+
• At least 4GB RAM available
• Ports 80, 443, 3000-3010, 5432, 6379, 9000-9001, 9090, 9200, 15672 available

Quick Start

1. Clone and Configure

# Create project directory
mkdir hub-spoke-architecture && cd hub-spoke-architecture

# Copy the docker-compose.yml file to this directory

# Create environment file
cp .env.example .env

# Edit .env with your secure passwords
nano .env

2. Create Required Directories

# Hub configuration
mkdir -p hub-config/ssl
mkdir -p hub-orchestrator

# Spoke services
mkdir -p spokes/auth-service
mkdir -p spokes/user-service
mkdir -p spokes/notification-service
mkdir -p spokes/analytics-service

# Monitoring configuration
mkdir -p monitoring/grafana/{dashboards,datasources}

# Database initialization scripts
mkdir -p init-scripts

3. Create Hub Gateway Configuration

Create hub-config/nginx.conf:

events {
    worker_connections 1024;
}

http {
    upstream hub_orchestrator {
        server hub-orchestrator:3000;
    }

    upstream spoke_auth {
        server spoke-auth:3000;
    }

    upstream spoke_user {
        server spoke-user:3000;
    }

    upstream spoke_notification {
        server spoke-notification:3000;
    }

    server {
        listen 80;
        server_name localhost;

        # Health check endpoint
        location /health {
            return 200 "OK\n";
            add_header Content-Type text/plain;
        }

        # Hub orchestrator
        location /api/hub/ {
            proxy_pass http://hub_orchestrator/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        # Authentication service
        location /api/auth/ {
            proxy_pass http://spoke_auth/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        # User service
        location /api/users/ {
            proxy_pass http://spoke_user/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        # Notification service
        location /api/notifications/ {
            proxy_pass http://spoke_notification/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

4. Create Database Initialization Script

Create init-scripts/create-multiple-databases.sh:

#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
    CREATE DATABASE auth_db;
    CREATE DATABASE user_db;
    CREATE DATABASE analytics_db;
EOSQL

Make it executable:

chmod +x init-scripts/create-multiple-databases.sh

5. Create Prometheus Configuration

Create monitoring/prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'hub-orchestrator'
    static_configs:
      - targets: ['hub-orchestrator:3000']
  
  - job_name: 'spoke-auth'
    static_configs:
      - targets: ['spoke-auth:3000']
  
  - job_name: 'spoke-user'
    static_configs:
      - targets: ['spoke-user:3000']
  
  - job_name: 'spoke-notification'
    static_configs:
      - targets: ['spoke-notification:3000']
  
  - job_name: 'spoke-analytics'
    static_configs:
      - targets: ['spoke-analytics:8000']

6. Start the Architecture

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Check service status
docker-compose ps

7. Verify Services

# Check hub gateway
curl http://localhost/health

# Check Redis
docker exec redis-broker redis-cli -a changeme ping

# Check RabbitMQ management UI
open http://localhost:15672  # admin/changeme

# Check Grafana
open http://localhost:3010  # admin/admin

# Check MinIO console
open http://localhost:9001  # minioadmin/minioadmin

Service Endpoints

Service	Port	Endpoint	Description
Hub Gateway	80	http://localhost	Main entry point
Hub Orchestrator	3000	http://localhost:3000	Central coordinator
Auth Service	3001	http://localhost:3001	Authentication
User Service	3002	http://localhost:3002	User management
Notification Service	3003	http://localhost:3003	Notifications
Analytics Service	3004	http://localhost:3004	Analytics
PostgreSQL	5432	localhost:5432	Database
Redis	6379	localhost:6379	Cache/Broker
RabbitMQ	5672	localhost:5672	Message Queue
RabbitMQ UI	15672	http://localhost:15672	Queue Management
MinIO	9000	http://localhost:9000	Object Storage
MinIO Console	9001	http://localhost:9001	Storage UI
Elasticsearch	9200	http://localhost:9200	Search Engine
Prometheus	9090	http://localhost:9090	Metrics
Grafana	3010	http://localhost:3010	Dashboards

Architecture Patterns

Communication Flow

Client Request
    ↓
Hub Gateway (NGINX)
    ↓
Hub Orchestrator (Brain)
    ↓
Message Broker (Redis/RabbitMQ)
    ↓
Spoke Services (Hands)
    ↓
Database/Storage

Hub Responsibilities (The Brain)

1. Request Routing: Determine which spoke should handle each request
2. Load Balancing: Distribute work across multiple spoke instances
3. State Management: Maintain session state and context
4. Service Discovery: Track available spokes and their health
5. Error Handling: Retry logic, circuit breaking, fallback strategies
6. Authentication: Validate tokens before routing to spokes
7. Rate Limiting: Protect spokes from overload
8. Logging & Metrics: Centralized observability

Spoke Responsibilities (The Hands)

1. Task Execution: Perform specific business logic
2. Data Processing: CRUD operations, transformations
3. External Integration: Call third-party APIs
4. Event Publishing: Notify hub of state changes
5. Health Reporting: Regular health checks to hub
6. Stateless Operation: No direct spoke-to-spoke communication

Scaling Strategies

Scale Individual Spokes

# Scale notification service to 3 instances
docker-compose up -d --scale spoke-notification=3

# Scale analytics service to 2 instances
docker-compose up -d --scale spoke-analytics=2

Add New Spokes

1. Define new service in docker-compose.yml
2. Connect to hub-network
3. Configure to communicate with hub-orchestrator
4. Add route in hub-config/nginx.conf
5. Restart gateway: docker-compose restart hub-gateway

Monitoring and Observability

View Metrics in Prometheus

open http://localhost:9090

Example queries:

• up{role="spoke"} - Check spoke health
• http_requests_total - Total requests
• http_request_duration_seconds - Request latency

Visualize in Grafana

open http://localhost:3010

1. Login with admin/admin
2. Add Prometheus datasource: http://prometheus:9090
3. Import dashboards for each service

View Logs

# All services
docker-compose logs -f

# Specific service
docker-compose logs -f hub-orchestrator

# Filter by role
docker-compose logs -f $(docker-compose ps -q | xargs docker inspect -f '{{if eq (index .Config.Labels "role") "spoke"}}{{.Name}}{{end}}')

Security Best Practices

1. Change Default Passwords: Update all passwords in .env
2. Use SSL/TLS: Configure SSL certificates in hub-config/ssl/
3. Network Isolation: Spokes can't communicate directly
4. Secret Management: Use Docker secrets in production
5. Least Privilege: Each service has minimal permissions
6. Regular Updates: Keep base images updated

Troubleshooting

Service Won't Start

# Check logs
docker-compose logs <service-name>

# Verify network connectivity
docker-compose exec hub-orchestrator ping spoke-auth

# Check environment variables
docker-compose config

Database Connection Issues

# Test PostgreSQL connection
docker-compose exec postgres-db psql -U postgres -c "SELECT 1"

# Check database exists
docker-compose exec postgres-db psql -U postgres -l

Redis Connection Issues

# Test Redis connection
docker-compose exec redis-broker redis-cli -a changeme ping

# Check Redis info
docker-compose exec redis-broker redis-cli -a changeme info

Production Deployment

Recommended Changes

1. Use Docker Swarm or Kubernetes for orchestration
2. External Load Balancer instead of NGINX container
3. Managed Databases (AWS RDS, Google Cloud SQL)
4. Secrets Management (HashiCorp Vault, AWS Secrets Manager)
5. Centralized Logging (ELK Stack, Splunk)
6. Auto-scaling based on metrics
7. Backup Strategy for databases and volumes

Environment-Specific Configs

# Development
docker-compose -f docker-compose.yml up

# Staging
docker-compose -f docker-compose.yml -f docker-compose.staging.yml up

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up

Maintenance

Backup Volumes

# Backup PostgreSQL
docker-compose exec postgres-db pg_dumpall -U postgres > backup.sql

# Backup Redis
docker-compose exec redis-broker redis-cli -a changeme --rdb /data/dump.rdb

Update Services

# Pull latest images
docker-compose pull

# Recreate services
docker-compose up -d --force-recreate

Clean Up

# Stop all services
docker-compose down

# Remove volumes (WARNING: deletes all data)
docker-compose down -v

# Remove images
docker-compose down --rmi all

Contributing

To add a new spoke service:

1. Create service directory in spokes/
2. Add service definition to docker-compose.yml
3. Connect to hub-network and spoke-network
4. Configure environment variables
5. Add route to hub-config/nginx.conf
6. Update this README

License

MIT License - feel free to use and modify for your projects.

Support

For issues and questions:

• Check logs: docker-compose logs -f
• Verify configuration: docker-compose config
• Review health checks: docker-compose ps