Competitive Programming Portfolio Platform

A production-ready data aggregation platform for competitive programming profiles across multiple platforms (LeetCode, Codeforces, CodeChef, AtCoder, CSES, HackerRank).

🏗️ Architecture

This system follows a microservices architecture with:

• FastAPI Backend: REST API with async workers
• PostgreSQL: Primary database for user data, submissions, problems
• Redis: Caching layer and task queue (RQ)
• React Frontend: Dashboard for viewing aggregated profiles
• Playwright + Requests: Web scraping with fallbacks
• Kubernetes: Container orchestration with Helm charts

🚀 Quick Start (Local Development)

Prerequisites

• Docker & Docker Compose
• Python 3.11+
• Node.js 18+
• PostgreSQL 15+

Setup

# Clone and setup
git clone <repo-url>
cd cp-portfolio-platform

# Copy environment file
cp .env.example .env

# Start services with Docker Compose
docker-compose up -d

# Install backend dependencies
cd backend
pip install -r requirements.txt

# Run database migrations
alembic upgrade head

# Seed sample data
python scripts/seed_data.py

# Install frontend dependencies
cd ../frontend
npm install

# Start development servers
npm run dev  # Frontend on :3000
cd ../backend && uvicorn app.main:app --reload  # Backend on :8000

Quick Test

# Test Codeforces scraper
curl http://localhost:8000/api/v1/users/codeforces/tourist

# View dashboard
open http://localhost:3000

📊 API Endpoints

Core Endpoints

• GET /api/v1/users/{platform}/{handle} - Get user profile
• POST /api/v1/users/{platform}/{handle}/sync - Trigger data sync
• GET /api/v1/users/{platform}/{handle}/submissions - Get submissions
• GET /api/v1/platforms - List supported platforms
• GET /api/v1/health - Health check

Authentication

• POST /api/v1/auth/login - Login with credentials
• POST /api/v1/auth/register - Register new account
• DELETE /api/v1/auth/logout - Logout

🏗️ Deployment

Kubernetes

# Deploy to K8s cluster
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/

# Or use Helm
helm install cp-platform ./helm/cp-platform

Production Checklist

• Set up managed PostgreSQL
• Configure Redis cluster
• Set up monitoring (Prometheus/Grafana)
• Configure ingress with TLS
• Set up CI/CD pipeline
• Configure secrets management

🔧 Adding New Platforms

1. Create scraper in backend/app/scrapers/new_platform.py
2. Implement the BaseScraper interface
3. Add platform configuration to app/config/platforms.py
4. Add tests in tests/scrapers/test_new_platform.py
5. Update documentation

📈 Monitoring & Observability

• Metrics: Prometheus metrics at /metrics
• Logs: Structured JSON logs
• Health: Health checks at /health
• Tracing: OpenTelemetry support

🔒 Security & Privacy

• Rate limiting per IP and user
• Input validation and sanitization
• HTTPS enforcement
• Secure session management
• GDPR compliance with data deletion
• Respects robots.txt

📚 Documentation

• Architecture
• API Reference
• Deployment Guide
• Security Guide
• Contributing

🧪 Testing

# Run all tests
pytest

# Run specific test suite
pytest tests/scrapers/
pytest tests/api/

# Run with coverage
pytest --cov=app tests/

📄 License

MIT License - see LICENSE file.