Skip to content
BranchesTags

Repository files navigation

Distributed Cache System

A high-performance, fault-tolerant distributed caching system built with Go and gRPC, featuring automatic failover, data replication, and service discovery using Consul.

Overview

This distributed cache system implements a multi-tier architecture designed to provide high availability, scalability, and fault tolerance for database query caching. The system uses a primary-replica replication model with automatic failover, ensuring data consistency and system resilience.

Architecture

┌─────────┐
│ Client  │
└────┬────┘
     │
     ▼
┌─────────────┐      ┌──────────────┐
│   Gateway   │◄────►│   Backend    │
│   Server    │      │    Server    │
└──────┬──────┘      └──────┬───────┘
       │                    │
       │                    ▼
       │              ┌──────────┐
       │              │  MySQL   │
       │              │ Database │
       │              └──────────┘
       ▼
┌──────────────┐
│Load Balancer │
└──────┬───────┘
       │
       ▼
┌─────────────────────────────────────┐
│        Cache Clusters               │
│  ┌──────────┐  ┌──────────┐        │
│  │ Primary  │  │ Primary  │  ...   │
│  │ (C0)     │  │ (C1)     │        │
│  └────┬─────┘  └────┬─────┘        │
│       │             │               │
│  ┌────▼─────┐  ┌───▼──────┐        │
│  │ Replica  │  │ Replica  │  ...   │
│  │ Replica  │  │ Replica  │        │
│  └──────────┘  └──────────┘        │
└────────────┬────────────────────────┘
             │
             ▼
      ┌────────────┐
      │  Sentinel  │
      │  (Monitor) │
      └────────────┘
             │
             ▼
      ┌────────────┐
      │   Consul   │
      │ (Discovery)│
      └────────────┘

Data Flow

  1. Query Request: Client sends JSON query to Gateway
  2. Cache Lookup: Gateway forwards to Load Balancer for cache check
  3. Cache Miss: On miss, Gateway queries Backend/MySQL
  4. Cache Population: Result is cached via Load Balancer
  5. Replication: Primary cache replicates to replicas (async/sync)
  6. Health Monitoring: Sentinel monitors all cache servers via heartbeat streams
  7. Failover: Automatic promotion of replica to primary on failure

Features

Core Features

  • Distributed Caching: Multi-cluster cache architecture with LRU eviction policy
  • Automatic Failover: Sentinel-based monitoring with automatic replica promotion
  • Data Replication: Configurable synchronous/asynchronous replication
  • Service Discovery: Consul-based dynamic service registration and discovery
  • Query Abstraction: JSON-based query interface for CRUD operations
  • Entity-based Invalidation: Invalidate all cache entries for a specific entity

Reliability Features

  • Graceful Shutdown: All services support graceful termination
  • Data Persistence: Cache state persisted to disk for recovery
  • Health Checks: Continuous health monitoring of all services
  • Heartbeat Streams: Real-time server health tracking via gRPC streams
  • Connection Pooling: Optimized database connection management

Performance Features

  • Load Balancing: Least-loaded server selection for cache operations
  • Consistent Hashing: Predictable cache key generation (SHA-256)
  • Parallel Operations: Concurrent cache and backend queries
  • LRU Cache: Efficient memory management with configurable size (200 entries/server)

Components

1. Gateway Server (Port: 8082)

Entry point for all client requests. Handles query parsing, cache coordination, and backend communication.

Key Responsibilities:

  • JSON query parsing and SQL generation
  • Cache hit/miss management
  • Backend health monitoring
  • Security validation (SQL injection prevention)

2. Backend Server (Port: 8080)

Manages MySQL database interactions and serves as the source of truth.

Key Responsibilities:

  • SQL query execution
  • Database connection pooling
  • Health check endpoints
  • Result serialization

3. Load Balancer (Port: 8084)

Distributes cache operations across cache servers and tracks query mappings.

Key Responsibilities:

  • Least-loaded cache server selection
  • Query-to-server mapping maintenance
  • Cache hit routing
  • Primary server metadata management

4. Cache Servers (Port: 9000+)

Distributed cache nodes organized in clusters with primary-replica topology.

Key Responsibilities:

  • LRU cache management (200 entries max)
  • Data replication (primary → replicas)
  • Heartbeat streaming to Sentinel
  • Offset tracking for incremental sync
  • Persistence to disk on shutdown

Cluster Structure:

  • Cluster ID = Server ID ÷ 4
  • Each cluster has 1 primary and N replicas
  • Automatic role determination via Sentinel

5. Sentinel (Port: 8081)

Monitors cache server health and orchestrates failover.

Key Responsibilities:

  • Cache server registration and role assignment
  • Heartbeat stream monitoring (3s intervals)
  • Failure detection (9s grace period)
  • Replica promotion based on offset (most up-to-date)
  • Load Balancer notification on topology changes

6. Client

Command-line interface for database operations.

Supported Operations:

  • create: Insert new records
  • read: Query existing records
  • update: Modify records
  • delete: Remove records

Prerequisites

  • Go: 1.25.0 or higher
  • MySQL: 5.7 or higher
  • Consul: Latest version
  • Operating System: Linux, macOS, or Windows

Installation

1. Clone the Repository

git clone <repository-url>
cd Distributed-Cache

2. Install Dependencies

go mod download

3. Start Consul

# Using Consul agent in development mode
consul agent -dev

4. Setup MySQL Database

# Create database and user
mysql -u root -p << EOF
CREATE DATABASE university;
CREATE USER 'cacheuser'@'localhost' IDENTIFIED BY '1234';
GRANT ALL PRIVILEGES ON university.* TO 'cacheuser'@'localhost';
FLUSH PRIVILEGES;
EOF

# Create sample table
mysql -u root -p university << EOF
CREATE TABLE student (
    id INT AUTO_INCREMENT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    program VARCHAR(100)
);
EOF

Configuration

Environment Variables

Variable Default Description
CACHE_BASE_PORT 9000 Starting port for cache servers
ACK_POLICY 0 Replication acknowledgment policy (0=async, N=wait for N replicas)
GATEWAY_PORT :8082 Gateway server port
BACKEND_PORT :8080 Backend server port
LB_PORT :8084 Load balancer port
SENTINEL_PORT :8081 Sentinel server port
DB_USER cacheuser MySQL username
DB_PASSWORD yourpassword MySQL password
DB_NAME university MySQL database name
DB_HOST 127.0.0.1:3306 MySQL host and port

Replication Policies

  • ACK_POLICY=0: Asynchronous replication (default, fastest)
  • ACK_POLICY=1: Wait for 1 replica acknowledgment
  • ACK_POLICY=N: Wait for N replica acknowledgments (max = number of replicas)

Usage

Starting the System

Start each component in separate terminals:

# 1. Start Backend Server
go run ./cmd/backend

# 2. Start Gateway Server
go run ./cmd/gateway

# 3. Start Load Balancer
go run ./cmd/loadbalancer

# 4. Start Sentinel
go run ./cmd/sentinel

# 5. Start Cache Servers (multiple instances)
# Cluster 0 (IDs 0-3)
go run ./cmd/cache --id=0  # Primary for Cluster 0
go run ./cmd/cache --id=1  # Replica
go run ./cmd/cache --id=2  # Replica
go run ./cmd/cache --id=3  # Replica

# Cluster 1 (IDs 4-7)
go run ./cmd/cache --id=4  # Primary for Cluster 1
go run ./cmd/cache --id=5  # Replica
# ... add more replicas as needed

Using the Client

Create Operation

go run ./cmd/client student create '{"first_name":"John","last_name":"Doe","program":"Computer Science"}'

Read Operation

# Read all students
go run ./cmd/client student read '{}'

# Read specific student
go run ./cmd/client student read '{"id":1}'

# Read by program
go run ./cmd/client student read '{"program":"Computer Science"}'

Update Operation

go run ./cmd/client student update '{"id":1,"program":"Data Science"}'

Delete Operation

go run ./cmd/client student delete '{"id":1}'

Custom Port Configuration

Start cache servers on custom ports:

go run ./cmd/cache --id=0 --port=9100

Testing Failover

  1. Start a cluster with primary and replicas
  2. Observe Sentinel heartbeat logs
  3. Kill the primary process (Ctrl+C or kill -9)
  4. Watch Sentinel detect failure and promote replica
  5. Verify Load Balancer receives notification
  6. Send queries to confirm system still works

API Reference

Gateway Service

ProcessQuery

Processes JSON-formatted database queries.

Request:

{
  "entity": "student",
  "operation": "read",
  "data": {"id": 1}
}

Response:

[
  {
    "id": 1,
    "first_name": "John",
    "last_name": "Doe",
    "program": "Computer Science"
  }
]

Supported Operations

CREATE

{
  "entity": "student",
  "operation": "create",
  "data": {
    "first_name": "Jane",
    "last_name": "Smith",
    "program": "Physics"
  }
}

READ

{
  "entity": "student",
  "operation": "read",
  "data": {"program": "Physics"},
  "select": "first_name,last_name"  // optional
}

UPDATE

{
  "entity": "student",
  "operation": "update",
  "data": {
    "id": 1,
    "program": "Mathematics"
  }
}

DELETE

{
  "entity": "student",
  "operation": "delete",
  "data": {"id": 1}
}

Cache Key Generation

Cache keys are generated using SHA-256 hash of:

entity:operation:sorted_json_data

Example:

student:read:{"id":1} → sha256 → e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Testing

Running Unit Tests

# Test cache implementation
go test ./internal/cache -v

# Test gateway server
go test ./internal/gateway -v

# Run all tests
go test ./... -v

Load Testing

Use the client in a loop to test system behavior:

# Read test
for i in {1..1000}; do
  go run ./cmd/client student read '{"id":1}'
done

# Write test
for i in {1..100}; do
  go run ./cmd/client student create "{\"first_name\":\"User$i\",\"last_name\":\"Test\",\"program\":\"CS\"}"
done

Failover Testing

# Script to test automatic failover
#!/bin/bash

# Start cluster
go run ./cmd/cache --id=0 &
PID_PRIMARY=$!
sleep 2
go run ./cmd/cache --id=1 &
go run ./cmd/cache --id=2 &

# Wait for registration
sleep 5

# Perform operations
go run ./cmd/client student create '{"first_name":"Test","last_name":"User","program":"CS"}'
go run ./cmd/client student read '{"first_name":"Test"}'

# Kill primary
kill -9 $PID_PRIMARY
echo "Primary killed, waiting for failover..."
sleep 10

# Test after failover
go run ./cmd/client student read '{"first_name":"Test"}'

Project Structure

.
├── cmd/                      # Main applications
│   ├── backend/             # Backend server entry point
│   ├── cache/               # Cache server entry point
│   ├── client/              # Client CLI
│   ├── gateway/             # Gateway server entry point
│   ├── loadbalancer/        # Load balancer entry point
│   └── sentinel/            # Sentinel entry point
├── internal/                # Internal packages
│   ├── backend/            # Backend implementation
│   ├── cache/              # Cache implementation
│   │   ├── lru/           # LRU cache algorithm
│   │   ├── cache.go       # Cache data structure
│   │   ├── server.go      # Cache gRPC server
│   │   └── replication.go # Replication logic
│   ├── gateway/           # Gateway implementation
│   ├── loadbalancer/      # Load balancer implementation
│   ├── sentinel/          # Sentinel implementation
│   └── pkg/               # Shared packages
│       ├── config/        # Configuration utilities
│       ├── consul/        # Consul client wrapper
│       └── graceful/      # Graceful shutdown utilities
├── protos/                # Protocol buffer definitions
├── gen/                   # Generated protobuf code
├── cache_data/           # Persistent cache storage
└── go.mod               # Go module definition

Key Design Decisions

1. LRU Cache with Fixed Size

  • Each cache server limited to 200 entries
  • Prevents unbounded memory growth
  • Automatic eviction of least recently used items

2. Primary-Replica Architecture

  • Ensures data durability through replication
  • Enables high availability via failover
  • Supports configurable consistency levels (ACK_POLICY)

3. Cluster-Based Organization

  • Cluster ID = Server ID ÷ 4
  • Allows horizontal scaling
  • Isolated failure domains

4. Heartbeat Streaming

  • Long-lived gRPC streams for health monitoring
  • 3-second heartbeat interval
  • 9-second grace period before failover
  • Includes offset for replica promotion selection

5. Consul for Service Discovery

  • Dynamic service registration
  • Automatic endpoint discovery
  • Eliminates hardcoded addresses
  • Supports service health checks

6. Incremental Replication

  • Backlog of last 10 operations
  • Replicas track offset for incremental sync
  • Falls back to full snapshot if too far behind
  • Minimizes network overhead

7. Entity-Based Invalidation

  • Maintains entity-to-query-hash index
  • Bulk invalidation on write operations
  • Ensures cache consistency after updates

Monitoring and Debugging

Log Messages

All components provide detailed logging:

[TIMESTAMP] CacheServer N: Assigned as PRIMARY for cluster X
[TIMESTAMP] Sentinel: Heartbeat received from primary N at HH:MM:SS
[TIMESTAMP] LoadBalancer: Processed 'read' operation on Primary CacheServer N
[TIMESTAMP] Gateway: Cache hit.

Health Indicators

  • Green: All heartbeats received on time
  • Yellow: Missed heartbeats but within grace period
  • Red: Grace period exceeded, failover initiated

Common Issues

Issue: Cache server won't start (port in use)

Solution: Server automatically tries next port. Check logs for actual port used.

Issue: Sentinel not detecting failures

Solution: Verify heartbeat stream is established. Check firewall rules.

Issue: Backend connection failed

Solution: Verify MySQL is running and credentials are correct.
Check DB_USER, DB_PASSWORD, DB_HOST environment variables.

Issue: Consul service discovery fails

Solution: Ensure Consul agent is running: consul agent -dev

Performance Characteristics

Throughput

  • Cache Hit: ~1-2ms latency
  • Cache Miss: ~10-50ms (depends on SQL query complexity)
  • Replication (Async): Negligible impact on write latency
  • Replication (Sync, N=1): +2-5ms per write

Capacity

  • Cache Servers: Each handles 200 queries (configurable)
  • Clusters: Unlimited (scale horizontally)
  • Replicas: Unlimited per cluster

Resource Usage

  • Memory: ~50MB per cache server
  • CPU: <5% per component under normal load
  • Network: ~1KB per query (cached), ~10KB (database)

Future Enhancements

  • Distributed consensus (Raft/Paxos) for multi-data-center deployment
  • Redis protocol compatibility
  • Metrics and monitoring dashboard (Prometheus/Grafana)
  • Cache warming strategies
  • Query result compression
  • Multi-region support with geo-replication
  • Write-behind caching
  • Configurable eviction policies (LFU, FIFO)
  • Read replicas for load distribution
  • Automated backup and restore

Contributing

Contributions are welcome! Please follow these guidelines:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Code Style

  • Follow Go conventions and idioms
  • Use gofmt for formatting
  • Add comments for exported functions
  • Include unit tests for new features

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

  • Built with gRPC for efficient RPC communication
  • Uses Consul for service discovery
  • Inspired by Redis clustering and Sentinel architecture
  • MySQL for persistent storage

Contact

For questions, issues, or suggestions, please open an issue on GitHub.

Version: 1.0.0
Last Updated: November 30, 2025

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages