Skip to content

Core Components

Phrimp edited this page Mar 30, 2025 · 5 revisions

Core Components

Consul: Service Discovery and Distributed Configuration

Consul serves as the backbone of our microservice architecture, providing:

  • Service Discovery: Enables microservices to find and communicate with each other without hardcoded addresses
  • Health Checking: Continuously monitors service health to ensure availability and facilitate failover
  • Key-Value Store: Centralized configuration storage that all services can access
  • Dynamic Configuration: Allows real-time updates to service configurations without restarts
  • DNS-based Service Discovery: Services can be discovered via simple DNS queries
  • Service Mesh Capabilities: Provides a foundation for advanced service-to-service communication patterns

In this implementation, Consul:

  • Maintains the service registry for all microservices
  • Stores Traefik's dynamic configurations including middleware definitions
  • Provides health status information for all registered services
  • Enables dynamic reconfiguration of the entire system
  • Acts as the source of truth for service locations and configurations

Traefik: Modern API Gateway and Edge Router

Traefik functions as the entry point and traffic manager for the system:

  • Automatic Service Discovery: Integrates with Consul to dynamically discover backend services
  • Middleware Pipeline: Processes requests through configurable middleware chains
  • Dynamic Configuration: Updates routing rules on-the-fly without restarts
  • Let's Encrypt Integration: Automatic SSL certificate provisioning and renewal
  • Circuit Breaking: Prevents cascading failures across services
  • Request Routing: Directs traffic to appropriate services based on paths, headers, and other criteria

In this implementation, Traefik:

  • Routes external client requests to appropriate internal microservices
  • Applies middleware for security, rate limiting, and request modification
  • Handles SSL termination for secure communication
  • Provides load balancing across service instances
  • Exposes a dashboard for monitoring and troubleshooting
  • Fetches configuration dynamically from Consul

HashiCorp Vault: Secret Management and Security

Vault provides enterprise-grade security features for the microservice ecosystem:

  • Secure Secret Storage: Centralized repository for all sensitive information
  • Dynamic Secrets: Generate temporary credentials with automatic expiration
  • Encryption as a Service: Provide encryption capabilities without exposing keys
  • Identity-based Access: Fine-grained control over who can access which secrets
  • Credential Rotation: Automatic rotation of credentials to enhance security
  • Audit Logging: Comprehensive logs of all secret access attempts

In this implementation, Vault:

  • Securely stores API keys, database credentials, and other sensitive information
  • Provides dynamic access credentials to services based on their identity
  • Implements the AppRole auth method for service-to-service authentication
  • Maintains policies that control which services can access which secrets
  • Uses Consul as its storage backend for high availability
  • Automates the loading of secrets from configuration files

Architecture Flow

Request Flow

  1. Client Request: A client makes an HTTP/HTTPS request to the system.
  2. Traefik Processing:
    • The request hits Traefik, which acts as the API gateway and entry point.
    • Traefik has already loaded routing rules and middleware configurations from Consul.
    • Traefik applies configured middlewares (rate limiting, authentication, etc.) directly to the request.
  3. Request Forwarding: After middleware processing, Traefik forwards the request to the appropriate microservice based on routing rules.
  4. Service Response: The microservice processes the request and returns a response.
  5. Response Delivery: Traefik forwards the response back to the client, potentially applying response middlewares.

Service Registration Flow

  1. Service Startup: When a microservice starts, it registers itself with Consul either:
    • Automatically via Docker labels (for containerized services)
    • Programmatically via Consul API (for services implementing direct registration)
  2. Health Checks: Consul performs regular health checks to ensure the service is operational.
  3. Service Discovery: Traefik and other services discover services through Consul's catalog or KV store.

Service-to-Service Communication

  1. Service Discovery: Microservice A queries Consul to discover the location of Microservice B.
  2. Direct Communication: After discovery, Microservice A communicates directly with Microservice B.
  3. Communication Methods:
    • REST/HTTP calls between services
    • gRPC for more efficient service-to-service communication
    • Event-based communication via message brokers (e.g., RabbitMQ)

Services do not communicate through Consul itself; they use Consul only to discover where other services are located. The actual communication happens directly between services.

Configuration Flow

  1. Infrastructure Setup: When the infrastructure starts, middleware configurations are registered in Consul KV store.
  2. Traefik Configuration:
    • Traefik loads initial configurations from Consul.
    • Traefik watches for changes to configurations in Consul and updates dynamically.
  3. Middleware Application:
    • Middlewares are configured in Consul and applied by Traefik during request processing.
    • Microservices reference these middlewares using labels or tags.
    • The actual middleware logic executes within Traefik, not as separate services.

Secret Management Flow

  1. Vault Initialization: On first startup, Vault is initialized and unsealed with encryption keys securely stored.
  2. Secret Loading: Predefined secrets are loaded into Vault's key-value store.
  3. Service Authentication: Microservices authenticate to Vault using the AppRole method.
  4. Secret Access:
    • Services request specific secrets they need from Vault.
    • Vault checks policies to ensure the service has appropriate access rights.
    • If approved, Vault provides the requested secrets to the service.
  5. Secret Rotation: Credentials can be automatically rotated without service disruption.

Prerequisites

  • Docker and Docker Compose installed
  • Git installed
  • Basic understanding of containerization and microservices
  • Free ports as specified in the .env file (by default: 80, 443, 8080, 8500, 8600, 8200)
  1. Add your services (see [Adding Services](Adding-Services)) [Architecture Overview](Architecture-Overview) - Detailed explanation of the architecture design [Core Components](Core-Components) - Information about Consul, Traefik, and Vault [Detailed Setup](Detailed-Setup) - Step-by-step setup instructions [Adding Services](Adding-Services) - How to add microservices to the architecture [Service Registration](Service-Registration) - Methods for registering services with Consul [Service Routing](Service-Routing) - Configuring Traefik for routing to services [Secret Management](Secret-Management) - Working with Vault for secrets [Health Checks](Health-Checks) - Implementing and monitoring service health [Advanced Features](Advanced-Features) - Advanced capabilities of each component [Production Deployment](Production-Deployment) - Checklist and best practices for production

Clone this wiki locally