Local Development Infrastructure

A generic Docker Compose setup providing common infrastructure services for local development environments. This README is generated by LLM.

📋 Overview

This setup provides a complete local development environment with:

• PostgreSQL 16 - Primary database with pre-configured application databases
• Redis Cluster - High-availability Redis setup with Sentinel
• Redis Standard - Single Redis instance for simple caching
• Apache Kafka - Event streaming platform with Zookeeper
• Kafka UI - Web interface for Kafka management

🚀 Quick Start

Prerequisites

• Docker Desktop or Docker Engine
• Docker Compose v2.0+

Start All Services

# Start all services
docker-compose up -d

# View running services
docker-compose ps

# Check logs
docker-compose logs -f

Stop Services

# Stop all services
docker-compose down

# Stop and remove volumes (⚠️ This will delete all data)
docker-compose down -v

🗄️ Services & Ports

Service	Port	Description	Container Name
PostgreSQL	5432	Primary database	local-postgres
Redis Master	6379	Redis master instance	local-redis-sentinel-master
Redis Slave 1	6380	Redis replica instance	local-redis-sentinel-slave1
Redis Slave 2	6381	Redis replica instance	local-redis-sentinel-slave2
Redis Sentinel	26379	Sentinel monitoring service	local-redis-sentinel1
Redis Standard	16379	Single Redis instance	local-redis-standard
Zookeeper	2181	Kafka's dependency service	local-zookeeper
Kafka	9094	Message broker	local-kafka
Kafka UI	9090	Kafka web interface	local-kafka-ui

🗄️ Database Configuration

PostgreSQL

Connection Details:

• Host: localhost
• Port: 5432
• User: local
• Password: local
• Database: postgres (default)

Pre-configured Databases:

• call-gate
• sb-internal
• sb-internal-test

Adding New Databases: Edit init-db.sql and add:

CREATE DATABASE "your-new-database";

Restart PostgreSQL to initialize:

docker-compose restart local-postgres

🔴 Redis Configuration

Redis Cluster with Sentinel (High Availability)

Master Instance:

• Host: localhost
• Port: 6379
• Config: Memory limit 4GB, CPU limit 2 cores

Slave Instances:

• Slave 1: localhost:6380
• Slave 2: localhost:6381
• Config: Memory limit 1GB, CPU limit 1 core each

Sentinel:

• Host: localhost
• Port: 26379
• Monitors master and handles automatic failover

Redis Standard Instance

Single Instance:

• Host: localhost
• Port: 16379
• No persistence, minimal configuration
• Ideal for development and testing

Connection Examples

Redis CLI:

# Connect to master
redis-cli -h localhost -p 6379

# Connect to slave
redis-cli -h localhost -p 6380

# Connect to standard instance
redis-cli -h localhost -p 16379

# Connect to Sentinel
redis-cli -h localhost -p 26379

Python (redis-py):

import redis

# Master/Slave configuration
master = redis.Redis(host='localhost', port=6379, decode_responses=True)
slave1 = redis.Redis(host='localhost', port=6380, decode_responses=True)

# Standard instance
redis_standard = redis.Redis(host='localhost', port=16379, decode_responses=True)

📊 Kafka Configuration

Zookeeper

Connection:

• Host: localhost
• Port: 2181
• Required by Kafka for cluster management

Kafka Broker

Configuration:

• Broker ID: 1
• Internal: local-kafka:9092
• External: localhost:9094
• Replication Factor: 1 (development only)
• Partitions: Default Kafka settings

Kafka UI

Access:

• URL: http://localhost:9090
• Cluster Name: local
• Bootstrap Servers: local-kafka:9092
• Zookeeper: local-zookeeper:2181

Basic Kafka Operations

Create Topic:

docker-compose exec local-kafka kafka-topics --create \
  --topic test-topic \
  --bootstrap-server localhost:9094 \
  --partitions 3 \
  --replication-factor 1

List Topics:

docker-compose exec local-kafka kafka-topics --list \
  --bootstrap-server localhost:9094

Send Messages:

docker-compose exec local-kafka kafka-console-producer \
  --topic test-topic \
  --bootstrap-server localhost:9094

Consume Messages:

docker-compose exec local-kafka kafka-console-consumer \
  --topic test-topic \
  --from-beginning \
  --bootstrap-server localhost:9094

🔧 Development Usage Examples

Environment Variables

Set these in your application's .env file:

# PostgreSQL
DATABASE_URL=postgresql://local:local@localhost:5432/postgres

# Redis configurations
REDIS_MASTER_URL=redis://localhost:6379
REDIS_SLAVE_1_URL=redis://localhost:6380
REDIS_SLAVE_2_URL=redis://localhost:6381
REDIS_SENTINEL_URL=redis://localhost:26379
REDIS_STANDARD_URL=redis://localhost:16379

# Kafka
KAFKA_BOOTSTRAP_SERVERS=localhost:9094
ZOOKEEPER_HOST=localhost:2181

Docker Compose Commands

# Start specific service
docker-compose up -d local-postgres

# Restart specific service
docker-compose restart local-kafka

# View service logs
docker-compose logs -f local-redis-sentinel-master

# Execute command in container
docker-compose exec local-postgres psql -U local -d postgres

# Scale services (for load testing)
docker-compose up -d --scale local-redis-sentinel-slave1=2

📁 File Structure

.
├── docker-compose.yaml    # Main orchestration file
├── init-db.sql           # Database initialization scripts
├── sentinel.conf         # Redis Sentinel configuration
└── README.md            # This file

⚙️ Customization

Adjust Resource Limits

Edit resource limits in docker-compose.yaml:

deploy:
  resources:
    limits:
      cpus: "2.0"      # Adjust CPU limit
      memory: 4g       # Adjust memory limit

Add New Services

Add new services to docker-compose.yaml following existing patterns.

Persistent Data

Data is persisted in Docker volumes. To reset all data:

docker-compose down -v
docker-compose up -d

🚨 Important Notes

Development Only

This configuration is designed for development only. For production:

• Increase replication factors
• Configure proper security settings
• Use persistent volumes for data
• Implement proper monitoring and logging

Resource Requirements

Minimum System Requirements:

• 8GB RAM (16GB recommended)
• 4 CPU cores (8 recommended)
• 20GB free disk space

Security

All services use development defaults with:

• No authentication
• Open network access
• Minimal security configurations

🐛 Troubleshooting

Common Issues

Port conflicts:

# Check if ports are in use
lsof -i :5432
lsof -i :6379
lsof -i :9094

# Stop conflicting services or change ports in docker-compose.yaml

Container won't start:

# Check container logs
docker-compose logs local-postgres
docker-compose logs local-kafka

# Restart specific service
docker-compose restart local-postgres

Database connection issues:

# Ensure PostgreSQL is ready
docker-compose exec local-postgres pg_isready -U local

# Test connection
docker-compose exec local-postgres psql -U local -d postgres

Redis connection issues:

# Test Redis connection
docker-compose exec local-redis-sentinel-master redis-cli ping

Reset Everything

# Complete reset (⚠️ Deletes all data)
docker-compose down -v
docker system prune -f
docker-compose up -d

🤝 Contributing

To add new infrastructure services:

1. Add service to docker-compose.yaml
2. Update ports configuration in this README
3. Add connection examples
4. Test the integration
5. Update environment variable examples

📝 License

This setup is provided as-is for local development purposes.

---

Happy Coding! 🚀