Autonomous Self-Healing Distributed Backend System

This project implements a complete feedback loop for autonomous system remediation using Java, Spring Boot, Kafka, Redis, and PostgreSQL.

🚀 Getting Started

1. Prerequisites

• Docker & Docker Compose
• Java 17+
• Maven

2. Start Infrastructure

docker-compose up -d

This starts:

• Kafka: Event backbone
• Redis: Rate limiting & cooldowns
• PostgreSQL: Policy storage
• Prometheus: Metrics collection
• Grafana: Visualization (Admin:admin)

3. Start Services

Run each service in a separate terminal:

# Terminal 1
cd service-node && mvn spring-boot:run

# Terminal 2
cd anomaly-detector && mvn spring-boot:run

# Terminal 3
cd decision-engine && mvn spring-boot:run

# Terminal 4
cd action-executor && mvn spring-boot:run

---

🧪 Simulation Scenarios

1. Memory Leak (Remediation: RESTART_SERVICE)

Call this multiple times to trigger the anomaly:

curl "http://localhost:8081/debug/memory-leak?megabytes=200"

Expectation: anomaly-detector sees the memory jump, decision-engine picks the RESTART_SERVICE policy, and action-executor logs the execution.

2. Thread Exhaustion (Remediation: RESTART_SERVICE)

curl "http://localhost:8081/debug/thread-exhaustion?seconds=60"

Expectation: The 10-thread pool in the service becomes saturated, thread_count spikes, and the loop triggers.

3. CPU Spike (Remediation: SCALE_UP)

curl "http://localhost:8081/debug/cpu-spike?seconds=30"

---

📊 Monitoring

• Prometheus: http://localhost:9090
• Grafana: http://localhost:3000 (Admin:admin)

---

🛠 Tech Stack Details

• Concurrency: ThreadPoolExecutor for action isolation; ConcurrentHashMap for local state; Redis for distributed locks.
• Messaging: Kafka topics for raw-metrics, system-alerts, and healing-decisions.
• Fault Tolerance: Idempotency via ActionID (simulated) and Redis-based cooldowns.