KeyGrid

Developed as part of the Parallel and Distributed Systems course at the University of Algarve (UAlg).

---

Overview

KeyGrid is a distributed key-value storage system designed to operate across multiple nodes, addressing core distributed systems challenges such as fault tolerance, consistency, availability, and scalability.

---

Summary

• Features
• Architecture Diagram
• Prerequisites
• Installation and Usage
  • Via Docker Compose
  • Local Development
• REST API
• Frontend
• Open Frontend
• Tests
  • Unit tests
  • Load (k6)
• Contributing
• Author
• License

---

Features

• Full CRUD for key–value pairs (PUT, GET, DELETE) via JSON REST.
• Strong persistence with PostgreSQL.
• Optional cache and pub/sub with Redis.
• High availability: three replicated FastAPI nodes (ports 8000, 8001 and 8002).
• HTTP round-robin load balancing via Nginx.
• Automatic horizontal scalability with Docker Compose / Kubernetes.
• Resilience: automatic restarts and health checks.
• Front-end SPA built with Vue 3 + Vite + Pinia.
• Bootstrap scripts start.sh / start.bat.
• Documentation in Markdown and Swagger UI.
• Unit tests (Pytest & Jest) and load testing (k6).

---

Architecture Diagram

Show diagram

---

Prerequisites

• Docker ≥ 20.x
• Docker Compose ≥ 1.29.x
• Node.js ≥ 16.x (for npm run dev)
• (Optional) Python 3.8+ to run locally without Docker

---

Installation and Usage

Via Docker Compose

git clone https://github.com/SteveRochaDev/keygrid.git
cd KeyGrid

# Linux/macOS
chmod +x start.sh

# Start the full stack (Redis, PostgreSQL, pgAdmin, backends, nginx/frontend)
./start.sh

# Windows PowerShell
.\start.bat

• Frontend: http://localhost:8080
• API (Swagger): http://localhost:8000/api/docs
• pgAdmin: http://localhost:5050 (user: admin@admin.com / pass: admin)

To stop without removing data volumes:

docker-compose down --remove-orphans

---

Local Development

Back-end (FastAPI)

cd backend
python3 -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\Activate
pip install -r requirements.txt
uvicorn src.main:app --reload --port 8000

Front-end (Vue 3 + Vite)

cd frontend
npm install
npm run dev

Open http://localhost:5173 and interact with the UI; Vite proxies /api → http://localhost:8000.

---

REST API

See full manual in docs/api_manual.md

Health-check

GET /api/health
Response 200
{ "status": "ok" }

Insert/Update pair

PUT /api/key
Request JSON:
{
  "data": {
    "key": "<foo>",
    "value": "<bar>"
  }
}
Response: HTTP 200
{ "status": "success" }

Retrieve value

GET /api/key?key=<foo>
200 → { "data": { "value": "<bar>" } }
404 → { "detail": "Key not found" }

Delete key

DELETE /api/key?key=<foo>
200 → { "status": "success" }
404 → { "detail": "Key not found" }

---

Frontend

• Vue components: PutForm.vue, GetForm.vue, DeleteForm.vue display messages and a “Show more” option.
• Pinia store in src/store/kvStore.ts abstracts fetch() calls.
• Responsive layout with custom properties in src/styles/globals.css.
• SPA fallback and /api proxy configured in vite.config.ts.

---

Tests

Unit tests

• Back-end: Pytest in tests/api/test_api.py

  cd backend
  pytest -q

• Front-end: Jest / Vue Test Utils in tests/unit/*.spec.js

  cd frontend
  npm test

Load (k6)

npm install -g k6
k6 run tests/load_test.js

---

SPD – Fail-over Test

1. Start the entire stack (proxy + 3 backends + front):

   cd "C:\Users\steve\OneDrive\Ambiente de Trabalho\KeyGrid"
   docker-compose up -d --build

2. Check that the 3 backends are healthy:

   docker ps --filter name=keygrid-backend
   # keygrid-backend1   Up (healthy)
   # keygrid-backend2   Up (healthy)
   # keygrid-backend3   Up (healthy)

3. Base test (create and read the key “failover”):

   # PUT
   Invoke-RestMethod -Uri http://localhost:8000/api/key -Method PUT `
     -ContentType application/json `
     -Body (@{ data=@{ key='failover'; value='ok' } }|ConvertTo-Json)
   
   # GET → 200 + {"data":{"value":"ok"}}
   curl.exe --% -i "http://localhost:8000/api/key?key=failover"

4. Simulate failure of each node in sequence:

   # a) Stop backend1
   docker-compose stop backend1
   Invoke-RestMethod -Uri "http://localhost:8000/api/key?key=failover" -Method GET  # 200 OK
   
   # b) Stop backend2
   docker-compose stop backend2
   Invoke-RestMethod -Uri "http://localhost:8000/api/key?key=failover" -Method GET  # 200 OK
   
   # c) Stop backend3
   docker-compose stop backend3
   curl.exe --% -i "http://localhost:8000/api/key?key=failover"                # 502 Bad Gateway
   
   # d) Restore all nodes
   docker-compose start backend1 backend2 backend3
   Invoke-RestMethod -Uri "http://localhost:8000/api/key?key=failover" -Method GET  # 200 OK

---

Documentation

• Architecture Diagram
• REST API Manual
• Installation Guide
• Load Test
• Distributed Systems Concepts
• Limits and Capacities
• Bibliography
• Requirements vs Implementation
• General Rules

---

Contributing

1. Fork this repository
2. Create a feature branch: git checkout -b feature/xyz
3. Commit your changes: git commit -m "feat: description"
4. Push to your fork: git push origin feature/xyz
5. Open a Pull Request

---

Author

This project was developed individually as part of academic coursework, covering system design, implementation, testing, and deployment.

---

License

This project is licensed under the MIT License.

---

Developed by Steve Rocha
University of Algarve — May 2025