Skip to content
BranchesTags

Latest commit

 

History

7 Commits

Folders and files

NameName
Last commit message
Last commit date
.gitignore
.gitignore
 
 
.python-version
.python-version
 
 
README.md
README.md
 
 
logo.png
logo.png
 
 
main.py
main.py
 
 
pyproject.toml
pyproject.toml
 
 

Repository files navigation

🚀 MyStack: Full-Stack AI Engineering Journey

MyStack Logo

From Simple Chatbot API → Enterprise RAG Platform

A Complete Learning Journey Through Six Production Layers

License: MIT Python Docker UV Open in GitHub Codespaces Open in Dev Containers

📑 Quick Navigation

⚠️ IMPORTANT DISCLAIMER

🚧 ACTIVE CONSTRUCTION ZONE 🚧

This repository is currently under heavy development!

  • 🔧 Work in Progress: Many features are incomplete or experimental
  • ⚡ Breaking Changes: APIs and structures may change without warning
  • 🧪 Experimental Code: Some components are proof-of-concepts
  • 🐛 Bugs Expected: This is a learning journey, not production-ready code
  • 🎯 Learning Focus: Primary goal is education, not stability

HACK AT YOUR OWN RISK! Better yet - CONTRIBUTE! 🛠️

🎯 TL;DR

A hands-on, layer-by-layer journey building a single chatbot into a production RAG platform. Learn full-stack AI engineering through six iterative versions, from FastAPI to Kubernetes. Not just a chatbot tutorial – learn universal production AI principles.

🎯 Who Is This For?

Choose Your Learning Track

👨‍💻 Backend Engineer Track

Start: Layer 1
Focus: API design, async patterns, database migrations
Path: Layers 1 → 2 → 3 → 6
Goal: Master AI system architecture and deployment

🔬 ML Practitioner Track

Start: Layer 2
Focus: RAG, vector search, model serving
Path: Layers 2 → 3 → 5
Goal: Bridge the gap between research and production

☁️ DevOps Track

Start: Layer 4
Focus: Automation, orchestration, scaling
Path: Layers 4 → 5 → 6
Goal: Master MLOps and cloud deployment

What You'll Learn:

Skill Category Layer Focus Real-World Application
API Design Layer 1 Building production-ready AI APIs
Vector Databases Layer 2 Semantic search and RAG systems
Database Architecture Layer 3 Choosing and migrating between databases
Async Processing Layer 4 Background jobs and task queues
MLOps & Monitoring Layer 5 Production observability and CI/CD
Cloud Deployment Layer 6 Kubernetes and distributed systems

Prerequisites:

Technical Skills:

  • ✅ Python (comfortable with async/await)
  • ✅ Basic ML/DL (train/test/inference concepts)
  • ✅ APIs (REST fundamentals)
  • ✅ Git (version control basics)

Helpful but Not Required:

  • 📚 Docker/containerization experience
  • ☁️ Cloud platform familiarity (AWS/GCP/Azure)
  • 🐧 Linux command line basics

⚠️ This is NOT an ML tutorial. We focus on the engineering, infrastructure, and production deployment of AI systems, assuming you already understand model fundamentals.

📁 Project Structure

MyStack/
├── 📁 layer1_basic_api/                    # Layer 1: Basic Chatbot API
├── 📁 layer2_rag_capability/               # Layer 2: RAG Capability
├── 📁 layer3_database_migration/           # Layer 3: Database Migration
├── 📁 layer4_automation_orchestration/     # Layer 4: Automation
├── 📁 layer5_mlops_monitoring/             # Layer 5: MLOps & Monitoring
├── 📁 layer6_production_scale/             # Layer 6: Production Scale
├── 📁 shared/                              # Shared utilities across layers
├── 📁 docs/                                # Documentation
├── 📁 scripts/                             # Development scripts
├── 📁 examples/                            # Example implementations
├── 📁 config/                              # Configuration files
├── 📁 .github/                             # GitHub workflows
├── 📁 .devcontainer/                       # Dev container configuration
├── 📁 .vscode/                             # VSCode settings
├── pyproject.toml                          # Project metadata & dependencies
├── uv.lock                                 # UV lock file
├── .env.example                            # Environment variables template
├── .gitignore                              # Git ignore file
├── .dockerignore                           # Docker ignore file
├── LICENSE                                 # MIT License
├── CONTRIBUTING.md                         # Contribution guidelines
├── CODE_OF_CONDUCT.md                      # Code of conduct
└── README.md                               # This file

Detailed Structure by Layer:

Layer 1: Basic Chatbot API

layer1_basic_api/
├── 📁 api/
│   ├── main.py              # FastAPI app
│   ├── config.py            # Configuration
│   ├── 📁 routes/           # API endpoints
│   ├── 📁 services/         # Business logic
│   ├── 📁 schemas/          # Pydantic models
│   ├── 📁 utils/            # Utilities
│   └── 📁 middleware/       # Middleware
├── 📁 tests/                # Tests
├── Dockerfile              # Docker configuration
├── docker-compose.yml      # Docker Compose setup
├── requirements.txt        # Python dependencies
├── pyproject.toml          # Modern Python packaging
├── .env.example            # Environment template
└── README.md               # Layer-specific README

Layer 2: RAG Capability

layer2_rag_capability/
├── 📁 api/
│   ├── 📁 routes/           # Enhanced endpoints
│   ├── 📁 services/         # RAG services
│   ├── 📁 schemas/          # Document schemas
│   └── 📁 models/           # MongoDB models
├── 📁 vector_store/         # Qdrant integration
├── 📁 document_processing/  # Document parsing & chunking
├── Dockerfile
├── docker-compose.yml      # Adds MongoDB + Qdrant
└── README.md

Layer 3: Database Migration

layer3_database_migration/
├── 📁 migrations/           # Alembic migrations
├── 📁 api/
│   ├── 📁 repositories/     # SQLModel repositories
│   ├── 📁 models/           # PostgreSQL models
│   └── 📁 services/         # Migration logic
├── 📁 scripts/              # Migration scripts
├── Dockerfile
├── docker-compose.yml      # PostgreSQL + pgvector
└── README.md

Layer 4: Automation & Orchestration

layer4_automation_orchestration/
├── 📁 celery_app/           # Celery configuration & tasks
├── 📁 airflow/              # Airflow DAGs
├── 📁 api/
│   └── 📁 routes/           # Job status endpoints
├── Dockerfile
├── docker-compose.yml      # Adds Redis, Celery, Airflow
└── README.md

Layer 5: MLOps & Monitoring

layer5_mlops_monitoring/
├── 📁 mlflow/               # Experiment tracking
├── 📁 monitoring/           # Prometheus + Grafana
├── 📁 ci_cd/                # GitHub Actions workflows
├── 📁 api/
│   ├── 📁 routes/           # Metrics & experiments endpoints
│   └── 📁 middleware/       # Request monitoring
├── Dockerfile
├── docker-compose.yml      # Adds MLflow, Prometheus, Grafana
└── README.md

Layer 6: Production Scale

layer6_production_scale/
├── 📁 kubernetes/           # K8s manifests & scripts
├── 📁 ray_serve/            # Distributed model serving
├── 📁 multi_tenant/         # Multi-tenant architecture
├── 📁 api/
│   └── 📁 routes/           # Tenant management endpoints
├── Dockerfile              # Multi-stage builds
├── docker-compose.yml      # Full production stack
├── helm/                   # Helm charts
└── README.md

🛠️ Prerequisites & System Setup

Minimum Requirements

Component Minimum Recommended
OS Windows (WSL2), macOS, Linux Linux (Ubuntu 22.04+)
RAM 8 GB 16 GB+ (32 GB for local LLMs)
Storage 20 GB free 50 GB+ free
Python 3.9+ 3.11+
Docker Docker Desktop Docker Engine + Compose

Hardware Recommendations

For Layers 1-3:

  • Any modern computer
  • 16 GB RAM for smooth operation
  • SSD storage recommended

For Layers 4-6:

  • Consider cloud development (AWS/GCP/Azure)
  • GPU optional until Layer 6
  • Multi-core CPU for parallel processing

Software Setup

  1. Install Docker & Docker Compose:

    # Ubuntu/Debian
sudo apt-get update
sudo apt-get install docker.io docker-compose

# macOS (with Homebrew)
brew install --cask docker

# Windows
Download Docker Desktop from docker.com

  2. Install Python & UV:

    # Install Python 3.11+
sudo apt install python3.11 python3.11-venv  # Ubuntu

# Install UV
curl -LsSf https://astral.sh/uv/install.sh | sh

🏗️ System Layers Overview

┌─────────────────────────────────────────────────────────┐
│    LAYER 6: PRODUCTION SCALE                            │
│    Kubernetes • Ray Serve • Multi-Tenant                │
│    Chatbot Platform v6.0 - Enterprise Scale             │
└────────────────────────┬────────────────────────────────┘
                         │ builds on
┌────────────────────────▼────────────────────────────────┐
│    LAYER 5: MLOps & MONITORING                          │
│    MLflow • Prometheus • Grafana • GitHub Actions       │
│    Chatbot Platform v5.0 - Full Observability           │
└────────────────────────┬────────────────────────────────┘
                         │ builds on
┌────────────────────────▼────────────────────────────────┐
│    LAYER 4: AUTOMATION & ORCHESTRATION                  │
│    Celery • Airflow • Background Jobs                   │
│    Chatbot Platform v4.0 - Automated Data Ingestion     │
└────────────────────────┬────────────────────────────────┘
                         │ builds on
┌────────────────────────▼────────────────────────────────┐
│    LAYER 3: DATABASE MIGRATION                          │
│    PostgreSQL • pgvector • SQLModel • Alembic           │
│    Chatbot Platform v3.0 - Production Database          │
└────────────────────────┬────────────────────────────────┘
                         │ builds on
┌────────────────────────▼────────────────────────────────┐
│    LAYER 2: RAG CAPABILITY                              │
│    MongoDB • Qdrant • Motor • Embeddings                │
│    Chatbot Platform v2.0 - RAG-Powered                  │
└────────────────────────┬────────────────────────────────┘
                         │ builds on
┌────────────────────────▼────────────────────────────────┐
│    LAYER 1: BASIC API                                   │
│    FastAPI • Docker • Ollama/HuggingFace                │
│    Chatbot Platform v1.0 - Simple API                   │
└─────────────────────────────────────────────────────────┘

📊 Layer Evolution & Skills Matrix

Layer Status Code Docs Skills You'll Gain Success Metrics
1 🟢 Complete ✅ 📘 FastAPI, Docker, Async Design API <2s response, 10 concurrent users
2 🟡 In Progress 🔄 📗 RAG, Vector DBs, Embeddings 100+ docs, <100ms search
3 🔵 Planned ❌ 📕 PostgreSQL, pgvector, SQLModel Zero data loss migration
4 🔵 Planned ❌ 📓 Celery, Airflow, Redis 1000+ docs concurrently
5 🔵 Planned ❌ 📒 MLflow, Prometheus, CI/CD <10s dashboard updates
6 🔵 Planned ❌ 📙 Kubernetes, Ray Serve 10k+ users, <100ms latency

📈 Expected Outcomes by Layer

After Layer 1: You can build and containerize a production-grade LLM API with proper error handling and testing.

After Layer 2: You can implement a complete RAG pipeline with document ingestion, vector search, and source attribution.

After Layer 3: You can design database schemas for AI applications, perform migrations, and optimize vector queries.

After Layer 4: You can build async processing pipelines, schedule workflows, and handle background jobs at scale.

After Layer 5: You can implement full MLOps with experiment tracking, monitoring, and CI/CD for AI systems.

After Layer 6: You can deploy and manage multi-tenant AI applications on Kubernetes with auto-scaling and distributed serving.

⚡ Quick Start

🚀 One-Liner Absolute Minimum

# Clone and start Layer 1 in one command
git clone https://github.com/yourusername/MyStack.git && cd MyStack && docker-compose up

🚀 Option 1: Full Docker Setup (Recommended)

# Clone the repository
git clone https://github.com/yourusername/MyStack.git
cd MyStack

# Start Layer 1 with Docker Compose
cd layer1_basic_api
docker-compose up --build

# Access the API (in 1-2 minutes)
# API Docs: http://localhost:8000/docs
# Health Check: http://localhost:8000/health

# Test the API
curl -X POST "http://localhost:8000/chat" \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, how are you?"}'

🚀 Option 2: Development Setup with UV

Prerequisites

  • Python 3.9 or higher
  • UV installed (recommended for optimal performance)
  • Docker & Docker Compose
  • Git

Installation

  1. Install UV (if not already installed):
# On macOS and Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# On Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
  1. Clone and setup the project:
# Clone the repository
git clone https://github.com/yourusername/MyStack.git
cd MyStack

# Create virtual environment and install dependencies
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install project with development dependencies
uv pip install -e ".[dev]"

# Or install production dependencies only
uv pip install -e .
  1. Start Layer 1: Basic Chatbot API:
cd layer1_basic_api

# Start Ollama (in separate terminal)
ollama pull llama3
ollama serve

# Run the API
uv run python -m uvicorn api.main:app --reload
# Server starts at http://localhost:8000
# API docs available at http://localhost:8000/docs

📦 Project Setup Commands

Command Description
uv venv Create a new virtual environment
uv pip install -e . Install the project in development mode
uv pip install -e ".[dev]" Install with development dependencies
uv pip install -e ".[test]" Install with testing dependencies
uv run python main.py Run the main application

🔧 Environment Configuration

  1. Copy the environment template:
cp .env.example .env
  1. Edit .env with your configuration:
# FastAPI Configuration
FASTAPI_HOST=0.0.0.0
FASTAPI_PORT=8000
FASTAPI_RELOAD=true

# Ollama Configuration
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=llama3

# Application Settings
DEBUG=true
LOG_LEVEL=INFO

🎯 The Evolution Story

One System. Six Layers. Complete Evolution.

You'll build ONE chatbot system that evolves from a simple API to a full-scale production RAG platform:

Layer 1: Simple chatbot API
    ↓ (add RAG)
Layer 2: + MongoDB + Qdrant vector search
    ↓ (migrate to better DB)
Layer 3: → PostgreSQL + pgvector + SQLModel
    ↓ (add automation)
Layer 4: + Celery + Airflow pipelines
    ↓ (add observability)
Layer 5: + MLOps + monitoring + CI/CD
    ↓ (scale to production)
Layer 6: + Kubernetes + distributed serving

Result: Production-ready RAG platform

The Learning Vehicle vs. The Destination

   Learning Vehicle:              Engineering Principles:
   ┌──────────────┐              ┌─────────────────────────┐
   │   Chatbot    │  → Teaches → │  Production AI Systems  │
   │  Journey     │              │                         │
   │              │              │ • Scalable Architecture │
   │ Layer 1: API │              │ • Vector Search         │
   │ Layer 2: RAG │              │ • Async Processing      │
   │ Layer 3: DB  │              │ • Model Serving         │
   │ ...          │              │ • Observability         │
   │ Layer 6: K8s │              │ • Multi-tenancy         │
   └──────────────┘              └─────────────────────────┘
   
   Specific Example                Universal Principles

🎯 Why a Chatbot-to-RAG Example?

A Practical, Transferable Learning Path

The chatbot → RAG platform journey is not just about building chatbots - it's a practical vehicle to learn engineering principles that apply to all AI applications.

Transferable Patterns You'll Learn:

Pattern Chatbot Example Other Applications
RAG Systems Document-based Q&A Knowledge bases, customer support, legal document analysis
Vector Search Semantic document search E-commerce recommendations, content discovery, fraud detection
Async Processing Background document ingestion Batch prediction jobs, data preprocessing, ETL pipelines
Model Serving LLM inference Computer vision models, recommendation systems, NLP classifiers
Multi-tenancy Multiple customer workspaces SaaS platforms, enterprise deployments, white-label solutions
Scalable APIs Chatbot API endpoints Prediction APIs, data services, integration endpoints

Universal AI Engineering Principles:

  1. 🔧 System Architecture

    • How to structure a production AI system
    • Choosing the right databases for different data types
    • Designing for scalability from day one

  2. ⚡ Performance Optimization

    • Model optimization techniques (ONNX, TensorRT)
    • Caching strategies for repeated queries
    • Async processing for background tasks

  3. 📊 Observability

    • Monitoring model performance in production
    • Tracking system health and user behavior
    • Implementing A/B testing for model iterations

  4. 🚀 Deployment Patterns

    • Containerization and orchestration
    • CI/CD for ML systems
    • Infrastructure as Code (IaC)

  5. 🔒 Production Concerns

    • Security and authentication
    • Rate limiting and quotas
    • Cost optimization and monitoring

The Real Goal:

"We're not just building a chatbot. We're learning to build production AI systems using a chatbot as our case study."

Every layer teaches principles that transfer directly to:

  • Computer Vision systems (replace RAG with image pipelines)
  • Recommendation engines (replace vectors with user embeddings)
  • Predictive Analytics (replace LLMs with regression models)
  • Any AI application requiring production deployment

🔰 LAYER 1: Basic Chatbot API

Start Simple: FastAPI + Docker + LLM

Duration: 1-2 weeks | Complexity: ⭐⭐☆☆☆

🎯 Goal

Build a working chatbot API that can answer questions using a local LLM (Ollama) or HuggingFace model.

🛠️ Technologies

  • FastAPI: REST API framework
  • Docker: Containerization
  • Ollama: Local LLM (Llama 3, Mistral, etc.)
  • HuggingFace: Alternative model serving
  • Pydantic: Data validation

📐 Architecture v1.0

┌──────────────────────────────┐
│   Simple Web UI / Postman    │
│   (Test interface)           │
└──────────────┬───────────────┘
               │ HTTP POST /chat
┌──────────────▼───────────────┐
│      FastAPI Application     │
│                              │
│  Routes:                     │
│  POST /chat                  │
│    - message: str            │
│    - returns: response       │
│                              │
│  GET /health                 │
│    - service status          │
│                              │
│  ┌────────────────────────┐  │
│  │  Chat Service          │  │
│  │  - Load Ollama model   │  │
│  │  - Generate response   │  │
│  │  - Simple prompting    │  │
│  └────────────────────────┘  │
└──────────────┬───────────────┘
               │
┌──────────────▼───────────────┐
│    Ollama (Port 11434)       │
│    - llama3:latest           │
│    - mistral:latest          │
└──────────────────────────────┘

📦 Capstone: Chatbot Platform v1.0

Core Features:

  • POST /chat: Send message, get AI response
  • GET /health: Check service health
  • Streaming responses (optional)
  • Basic error handling
  • Docker containerization

Project Structure:

layer1_basic_api/
├── api/
│   ├── main.py              # FastAPI app
│   ├── routes/
│   │   └── chat.py          # Chat endpoints
│   ├── services/
│   │   └── llm_service.py   # Ollama/HF integration
│   ├── schemas/
│   │   └── chat.py          # Pydantic models
│   └── config.py            # Settings
├── tests/
│   └── test_chat.py
├── docker-compose.yml
├── Dockerfile
├── requirements.txt
└── README.md

Success Metrics:

  • ✅ API responds <2 seconds
  • ✅ Handles 10 concurrent requests
  • ✅ Docker container runs successfully

🔍 LAYER 2: RAG Capability

Add Knowledge: MongoDB + Qdrant + RAG

Duration: 2-3 weeks | Complexity: ⭐⭐⭐☆☆

Builds on: Layer 1 (Chatbot v1.0)

🎯 Goal

Extend v1.0 to add RAG capabilities: upload documents, store them in MongoDB and Qdrant, and answer questions based on your knowledge base.

🛠️ Technologies Added

  • MongoDB: Document storage (with Motor for async)
  • Qdrant: Vector database for embeddings
  • Sentence Transformers: Generate embeddings
  • LangChain: RAG orchestration
  • PyPDF2 / python-docx: Document parsing

📐 Architecture v2.0

┌──────────────────────────────┐
│   Enhanced Web UI            │
│   + Document upload          │
│   + RAG-powered chat         │
└──────────────┬───────────────┘
               │
┌──────────────▼───────────────┐
│    FastAPI (Enhanced)        │
│                              │
│  NEW Routes:                 │
│  POST /documents/upload      │
│    - Upload PDF/DOCX/TXT     │
│  GET  /documents/list        │
│    - List all documents      │
│  DELETE /documents/{id}      │
│                              │
│  ENHANCED Route:             │
│  POST /chat                  │
│    - Now uses RAG!           │
│    - use_rag: bool           │
│                              │
│  ┌────────────────────────┐  │
│  │  Enhanced Services     │  │
│  │  + Document parser     │  │
│  │  + Embedding generator │  │
│  │  + RAG pipeline        │  │
│  │  + Vector search       │  │
│  └────────────────────────┘  │
└─────┬───────────────┬────────┘
      │               │
      │ (NEW)         │ (NEW)
      ▼               ▼
┌──────────────┐  ┌──────────────┐
│  MongoDB     │  │   Qdrant     │
│  (Motor)     │  │ (Port 6333)  │
│              │  │              │
│ Collections: │  │ Collections: │
│ - documents  │  │ - embeddings │
│ - chunks     │  │ - vectors    │
│ - metadata   │  │              │
└──────────────┘  └──────────────┘
      │
      │ (from Layer 1)
      ▼
┌──────────────┐
│   Ollama     │
└──────────────┘

Key Features Added:

  • Document upload (PDF, DOCX, TXT)
  • Automatic chunking and embedding
  • Vector similarity search
  • RAG-based chat responses
  • Document management (list, delete)

Success Metrics:

  • ✅ Upload and process 100+ documents
  • ✅ Vector search <100ms
  • ✅ RAG response <3 seconds
  • ✅ Source attribution accuracy >90%

🔄 LAYER 3: Database Migration

Production Database: PostgreSQL + pgvector + SQLModel

Duration: 2-3 weeks | Complexity: ⭐⭐⭐☆☆

Builds on: Layer 2 (Chatbot v2.0)

🎯 Goal

Migrate from MongoDB to PostgreSQL for better relational data handling, add pgvector for vector storage, and use SQLModel with Alembic for proper schema management.

Why Migrate?

  • PostgreSQL: ACID compliance, better for structured data
  • pgvector: Vectors + relational data in one DB (simplify architecture)
  • SQLModel: Type-safe ORM with Pydantic integration
  • Alembic: Professional schema migrations

📐 Architecture v3.0

┌──────────────────────────────┐
│   Web UI (unchanged)         │
└──────────────┬───────────────┘
               │
┌──────────────▼───────────────┐
│    FastAPI (Refactored)      │
│                              │
│  MIGRATED Services:          │
│  - SQLModel models           │
│  - Async PostgreSQL queries  │
│  - Type-safe database ops    │
│                              │
│  Routes (same endpoints):    │
│  POST /chat                  │
│  POST /documents/upload      │
│  GET  /documents/list        │
│                              │
│  ┌────────────────────────┐  │
│  │  Refactored Services   │  │
│  │  - SQLModel repository │  │
│  │  - pgvector queries    │  │
│  │  - Migration logic     │  │
│  └────────────────────────┘  │
└──────────────┬───────────────┘
               │
┌──────────────▼───────────────┐
│  PostgreSQL (Port 5432)      │
│  + pgvector extension        │
│                              │
│  Tables (SQLModel):          │
│  - documents                 │
│  - chunks                    │
│  - conversations             │
│  - messages                  │
└──────────────────────────────┘

(Qdrant removed - vectors now in PostgreSQL)
(MongoDB removed - data migrated to PostgreSQL)

Success Metrics:

  • ✅ All data migrated successfully
  • ✅ pgvector search as fast as Qdrant
  • ✅ Zero data loss during migration
  • ✅ Type safety with SQLModel
  • ✅ Proper foreign key relationships

⚡ LAYER 4: Automation & Orchestration

Automate Everything: Celery + Airflow + Background Jobs

Duration: 2-3 weeks | Complexity: ⭐⭐⭐⭐☆

Builds on: Layer 3 (Chatbot v3.0)

🎯 Goal

Add automation: Background document processing, scheduled data ingestion, and workflow orchestration with Airflow.

Why Automation?

  • Celery: Handle long-running tasks asynchronously
  • Airflow: Orchestrate complex data pipelines
  • Redis: Message broker and result backend
  • Automatic ingestion: Monitor folders, APIs, etc. for new documents

📐 Architecture v4.0

┌──────────────────────────────────────────────────────┐
│            FastAPI (Enhanced)                        │
│                                                      │
│  NEW Routes:                                         │
│  POST /jobs/process-document    (async)              │
│  GET  /jobs/{job_id}/status                          │
│  POST /documents/batch-upload   (async)              │
│                                                      │
│  ENHANCED:                                           │
│  POST /documents/upload → Now queues to Celery       │
└────────────────┬────────────────┬────────────────────┘
                 │                │
                 ▼                ▼
         ┌──────────────┐  ┌──────────────────┐
         │    Redis     │  │  PostgreSQL      │
         │  (Port 6379) │  │  (from Layer 3)  │
         │              │  └──────────────────┘
         │ - Task Queue │
         │ - Results    │
         └──────┬───────┘
                │ Consume tasks
┌───────────────▼──────────────────────────────┐
│           Celery Workers (Scaled)            │
│                                              │
│  ┌──────────────┐  ┌──────────────┐          │
│  │  Worker 1    │  │  Worker 2    │          │
│  │              │  │              │          │
│  │ Tasks:       │  │ Tasks:       │          │
│  │ - Parse docs │  │ - Generate   │          │ 
│  │ - Chunk text │  │   embeddings │          │
│  │ - Extract    │  │ - Index data │          │
│  └──────────────┘  └──────────────┘          │
└──────────────────────────────────────────────┘

┌──────────────────────────────────────────────┐
│         Apache Airflow (Port 8080)           │
│                                              │
│  DAGs (Scheduled Workflows):                 │
│                                              │
│  1. Daily Document Ingestion                 │
│     Schedule: Daily at 2 AM                  │
│                                              │
│  2. Weekly Embedding Refresh                 │
│     Schedule: Weekly on Sunday               │
│                                              │
│  3. Data Quality Checks                      │
│     Schedule: Hourly                         │
└──────────────────────────────────────────────┘

Success Metrics:

  • ✅ Process 1000+ documents concurrently
  • ✅ Workers handle 100 docs/minute
  • ✅ Job queue never exceeds 5 minutes
  • ✅ Airflow DAGs run successfully
  • ✅ Zero failed tasks (or <1%)

📈 LAYER 5: MLOps & Monitoring

Observability: MLflow + Prometheus + Grafana + CI/CD

Duration: 2-3 weeks | Complexity: ⭐⭐⭐⭐☆

Builds on: Layer 4 (Chatbot v4.0)

🎯 Goal

Add complete observability: Track experiments, monitor system health, implement CI/CD, and enable A/B testing.

📐 Architecture v5.0

┌──────────────────────────────────────────────────┐
│   Web UI + Grafana Dashboards Embedded           │
└──────────────────┬───────────────────────────────┘
                   │
┌──────────────────▼───────────────────────────────┐
│         FastAPI (Instrumented)                   │
│                                                  │
│  ALL Routes now tracked:                         │
│  - Request duration                              │
│  - Response codes                                │
│  - Token usage                                   │
│  - RAG performance                               │
│  - Cache hit rates                               │
│                                                  │
│  NEW Routes:                                     │
│  GET /metrics         (Prometheus endpoint)      │
│  GET /experiments     (MLflow experiments)       │
│  POST /models/ab-test (A/B testing)              │
└──────────┬─────────────────┬─────────────────────┘
           │                 │
           ▼                 ▼
    ┌─────────────┐   ┌──────────────┐
    │ Prometheus  │   │   MLflow     │
    │ (Port 9090) │   │ (Port 5000)  │
    └──────┬──────┘   └──────────────┘
           │
           ▼
    ┌─────────────┐
    │  Grafana    │
    │ (Port 3001) │
    │             │
    │ Dashboards: │
    │ - API perf  │
    │ - RAG stats │
    │ - System    │
    │ - Costs     │
    └─────────────┘

┌──────────────────────────────────────────────────┐
│         GitHub Actions CI/CD                     │
│                                                  │
│  On Push (main):                                 │
│  1. Run tests (pytest)                           │
│  2. Build Docker images                          │
│  3. Deploy to staging                            │
│  4. Run integration tests                        │
│                                                  │
│  On Tag (v*):                                    │
│  1. Deploy to production                         │
│  2. Register model in MLflow                     │
│  3. Create release notes                         │
└──────────────────────────────────────────────────┘

Success Metrics:

  • ✅ 100% of requests instrumented
  • ✅ Dashboards update <10s delay
  • ✅ CI/CD pipeline <10 minutes
  • ✅ Zero-downtime deployments
  • ✅ Alert latency <1 minute

🚀 LAYER 6: Production Scale

Enterprise Ready: Kubernetes + Ray Serve + Multi-Tenant

Duration: 3-4 weeks | Complexity: ⭐⭐⭐⭐⭐

Builds on: Layer 5 (Chatbot v5.0)

🎯 Goal

Transform to enterprise scale: Deploy on Kubernetes with auto-scaling, distributed model serving with Ray, multi-tenancy, and handle 10,000+ concurrent users.

📐 Architecture v6.0 (Production)

                   ┌─────────────────┐
                   │  Cloud Provider │
                   │  (AWS/GCP/Azure)│
                   └────────┬────────┘
                            │
                   ┌────────▼────────┐
                   │ Load Balancer   │
                   │ + SSL/TLS       │
                   └────────┬────────┘
                            │
┌───────────────────────────▼───────────────────────────┐
│           Kubernetes Cluster (Multi-node)             │
│                                                       │
│  ┌─────────────────────────────────────────────┐      │
│  │       NGINX Ingress Controller              │      │
│  │  Routes: /api /mlflow /grafana /airflow     │      │
│  └───┬────────────────────────┬────────────────┘      │
│      │                        │                       │
│  ┌───▼────────┐          ┌────▼──────────┐            │
│  │ API Pods   │          │ Worker Pods   │            │
│  │ (HPA 3-20) │          │ (HPA 5-50)    │            │
│  │            │          │               │            │
│  │ FastAPI    │          │ Celery        │            │
│  │ Replicas   │          │ Workers       │            │
│  └─────┬──────┘          └───────────────┘            │
│        │                                              │
│  ┌─────▼──────────────────────────────────┐           │
│  │     Ray Serve Cluster (GPU Nodes)      │           │
│  │                                        │           │
│  │  ┌──────────┐  ┌──────────┐            │           │
│  │  │ Llama 3  │  │ Mistral  │  More      │           │
│  │  │ (ONNX)   │  │(TensorRT)│  Models    │           │
│  │  └──────────┘  └──────────┘            │           │
│  │                                        │           │
│  │  Features:                             │           │
│  │  - Dynamic batching                    │           │
│  │  - Model composition                   │           │ 
│  │  - A/B testing                         │           │
│  │  - Multi-GPU                           │           │
│  └────────────────────────────────────────┘           │
└───────────────────────────────────────────────────────┘

Enterprise Features:

  • Multi-tenancy: Isolated workspaces per customer with quotas
  • Multiple optimized models: 3+ models with ONNX/TensorRT optimization
  • Auto-scaling: HPA for API pods and workers based on load
  • Distributed serving: Ray Serve for efficient model serving
  • 99.9% SLA: High availability setup

Success Metrics:

  • ✅ Handle 10,000+ concurrent users
  • ✅ API latency <100ms (p95)
  • ✅ Model inference <50ms
  • ✅ 99.9% uptime over 30 days
  • ✅ Auto-scale from 3 to 20 pods successfully
  • ✅ Support 100+ tenants

❓ Frequently Asked Questions (FAQ)

Q: Can I skip layers?

A: Yes! Each layer is designed to be semi-independent. Use the git tag for each version:

git checkout v2.0  # Jump to RAG implementation
git checkout v4.0  # Jump to automation

Q: Do I need a GPU?

A: Only for local model inference in Layers 1-2 (Ollama runs decently on CPU). Layers 5-6 discuss GPU optimization but can be simulated with CPU.

Q: How much will this cost to run in the cloud?

A: See our Cost Optimization section. Layer 1 can run on a free-tier VM (~$0-10/month). Layer 6 for production starts at ~$300/month.

Q: I'm new to Docker/Kubernetes. Can I still follow along?

A: Absolutely! We provide Docker Compose setups for each layer. Layer 6 (Kubernetes) includes detailed setup guides for beginners.

Q: What if I get stuck?

A:

  1. Check the docs/ folder for layer-specific guides
  2. Look for existing issues in GitHub
  3. Create a new issue with your question
  4. Consider contributing a fix or documentation improvement!

Q: Is this production-ready code?

A: No! This is a learning journey. The code is experimental and educational. Use it as a reference, not as production code without significant modification and testing.

🆘 Getting Help & Community

📣 Where to Ask Questions

  • GitHub Issues: For bugs, feature requests, and technical questions
  • Discussions: For design discussions and learning help
  • Twitter: Quick updates and announcements @yourusername

🚨 Common Issues & Solutions

Issue Solution
"Ollama connection refused" Run ollama serve in another terminal
"Docker out of memory" Increase Docker RAM to 8GB+ in settings
"UV not found" Re-run install script or check PATH
"Port already in use" Change ports in .env or stop conflicting services
"Module not found" Run uv pip install -e ".[dev]" again

🎯 How to Ask Good Questions

  1. Check the docs/troubleshooting.md first
  2. Include: OS, Python version, error logs
  3. Specify which layer you're working on
  4. Share what you've already tried
  5. Include relevant code snippets

🌐 Community Channels

  • Discord/Slack: Join our community for real-time help
  • Stack Overflow: Use tag [mystack] for questions
  • YouTube: Tutorial videos and walkthroughs
  • Newsletter: Updates on new layers and features

🔮 Beyond Layer 6: Next Steps

Continue Your Learning Journey

  • Advanced Topics: Fine-tuning, model distillation, edge deployment
  • Specializations:
    • ML Engineer: Model optimization, quantization, distillation
    • Infra Engineer: Service mesh, advanced K8s, multi-cloud
    • Data Engineer: Data pipelines, feature stores, data versioning
  • Real Projects: Fork this and build your own specialized version

Project Ideas Using This Foundation

  1. Legal Document Analyzer: RAG for legal research with case law citations
  2. Medical Q&A System: HIPAA-compliant medical assistant with patient data
  3. Enterprise Search: Company knowledge base with role-based access control
  4. Customer Support Bot: Multi-channel support with human handoff and sentiment analysis
  5. Academic Research Assistant: Paper analysis, citation generation, literature reviews
  6. Code Documentation Generator: Automatically document codebases with examples

Advanced Topics to Explore

  • Model Fine-tuning: Customize models for specific domains
  • Federated Learning: Train models across distributed data sources
  • Edge AI: Deploy models on edge devices (phones, IoT)
  • Multi-modal Systems: Combine text, image, and audio processing
  • Agent Systems: Autonomous agents that can take actions
  • LLM Ops: Specialized monitoring for large language models

Career Pathways

  • AI Engineer: Full-stack AI system development
  • MLOps Engineer: Focus on deployment and monitoring
  • AI Infrastructure Engineer: Cloud and infrastructure specialization
  • AI Product Manager: Technical product management for AI products
  • AI Consultant: Help companies implement AI solutions

🌟 Project Showcase

Built With MyStack

Project Description Layer Used Link
LegalRAG Legal document analysis system Layer 2-3 GitHub
EduBot Educational assistant for students Layer 1-2 GitHub
CompanyKB Internal company knowledge base Layer 4-5 GitHub
ResearchAssistant Academic paper analysis tool Layer 2-3 GitHub
CodeDocs AI Automated code documentation Layer 2-4 GitHub

Share Your Project!

Made something cool with MyStack? Add it here via PR!

  1. Fork the repository
  2. Add your project to the table above
  3. Submit a Pull Request with your addition

Featured Implementations

## 🏆 Community Spotlight

### Best Educational Project
**StudentHelper** by @learner123
- **What**: AI tutor for computer science students
- **Layers Used**: 1-3
- **Special Features**: Interactive coding exercises, progress tracking
- **Repo**: https://github.com/learner123/StudentHelper

### Most Innovative Use
**FarmAI** by @agritech
- **What**: Agricultural knowledge base for farmers
- **Layers Used**: 2-4  
- **Special Features**: Multilingual support, offline mode
- **Repo**: https://github.com/agritech/FarmAI

🧪 Testing Strategy

Testing Pyramid

                    ┌────────┐
                    │   E2E  │  ← Full system tests (slow, few)
                    └────────┘
                 ┌──────────────┐
                 │ Integration  │  ← API + DB + Model (medium)
                 └──────────────┘
            ┌────────────────────────┐
            │      Unit Tests        │  ← Individual functions (fast, many)
            └────────────────────────┘

Phase-Specific Testing

Layer 1:

# Unit tests: Models, utils, business logic
pytest tests/unit/

# Integration tests: API + Database
pytest tests/integration/

# Coverage target: 80%+
pytest --cov=api --cov-report=html

Layer 2:

# Add async tests
pytest tests/integration/test_celery_tasks.py

# Vector search accuracy tests
pytest tests/integration/test_vector_search.py

# End-to-end RAG pipeline
pytest tests/e2e/test_rag_pipeline.py

Layer 6:

# Kubernetes deployment tests
pytest tests/k8s/

# Load testing
locust -f tests/load/locustfile.py

# Model performance benchmarks
pytest tests/benchmarks/

💰 Cost Optimization

Resource Planning by Layer

Layer 1: Development (~$50/month)

  • Single VPS/EC2 instance (t3.medium)
  • PostgreSQL (managed or self-hosted)
  • Redis (ElastiCache free tier)
  • Total: $30-50/month

Layer 2: Small Production (~$200/month)

  • 2x API servers (load balanced)
  • Managed PostgreSQL + Redis
  • MongoDB Atlas (free tier → $25/month)
  • Qdrant Cloud (free tier → $50/month)
  • Celery workers (2-3 instances)
  • Total: $150-250/month

Layer 6: Enterprise Scale (~$1000+/month)

  • Kubernetes cluster (3-5 nodes)
  • GPU instances for inference (on-demand)
  • Managed databases
  • Monitoring stack
  • CDN and load balancer
  • Total: $1000-5000/month (highly variable)

Cost Optimization Strategies

  1. Use Spot/Preemptible Instances

    • 70-90% discount for Celery workers
    • Use for non-critical batch jobs

  2. Auto-scaling

    • Scale down during off-hours
    • Use Kubernetes HPA for demand-based scaling

  3. Caching Aggressively

    • Redis for repeated queries
    • CDN for static assets
    • Model output caching

  4. Optimize Model Inference

    • ONNX + TensorRT (3-5x faster)
    • Quantization (INT8 instead of FP32)
    • Batch requests together

🤝 Contributing

🚀 CONTRIBUTIONS WELCOME! 🚀

This is a learning journey, not a polished product!

We're building in public, embracing the messiness of learning. Your contributions, suggestions, and feedback are highly encouraged!

🎯 Why Contribute?

  • 🧠 Learn together: Share knowledge and grow as engineers
  • 🔧 Break things safely: Experimental environment welcome
  • 📚 Document the journey: Help others learn from our mistakes
  • 🚀 Build something amazing: Collective effort creates better results
  • 🤝 Join a community: Connect with other aspiring full-stack AI engineers

🛠️ Development Workflow

# 1. Fork and clone the repository
git clone https://github.com/yourusername/MyStack.git
cd MyStack

# 2. Set up development environment with UV
uv venv
source .venv/bin/activate
uv pip install -e ".[dev]"

# 3. Create a feature branch
git checkout -b feature/amazing-feature

# 4. Make your changes and commit
git commit -m "Add amazing feature"

# 5. Push to your fork
git push origin feature/amazing-feature

# 6. Create a Pull Request

💡 Areas Needing Help

Area Need How You Can Help
Layer 1 FastAPI/Ollama examples Create integration examples
Layer 2 RAG implementations Write RAG pipeline examples
Documentation Tutorials, guides Write docs, create tutorials
Testing Test coverage Add unit/integration tests
Examples Real-world use cases Create example projects
Bug Fixes Experimental code has bugs Find and fix issues
New Features Missing components Implement planned layers

📚 Learning Resources

Layer-Specific Guides

Layer 1:

  • docs/layer1/01-fastapi-basics.md
  • docs/layer1/02-ollama-integration.md
  • docs/layer1/03-docker-setup.md

Layer 2:

  • docs/layer2/01-rag-fundamentals.md
  • docs/layer2/02-mongodb-motor.md
  • docs/layer2/03-qdrant-setup.md
  • docs/layer2/04-embedding-strategies.md

Layer 3:

  • docs/layer3/01-why-migrate-postgres.md
  • docs/layer3/02-sqlmodel-guide.md
  • docs/layer3/03-pgvector-setup.md
  • docs/layer3/04-alembic-migrations.md

External Resources:

📊 Progress Log

📅 Date 🎯 Milestone 📝 Notes
2025-12-09 🎉 Repository Initialized Folder structure defined, starting with Layer 1
2025-12-09 ⚡ UV Integration Modern Python packaging with UV implemented
2025-12-09 🎨 Visual Assets Added logo.png and stack.png for branding
2025-12-09 ⚠️ Warning Added Clear disclaimer about experimental nature
Coming Soon 🐳 Docker Setup Containerization of Layer 1 services
Coming Soon 📊 Database Layer PostgreSQL + Redis integration
Coming Soon 🔄 CI/CD Pipeline GitHub Actions workflow setup

🎓 The Big Picture: AI Engineering Mastery

┌─────────────────────────────────────────────────────────────┐
│          From Principles to Production Applications         │
│                                                             │
│  Principles Learned:             →  Applications Possible:  │
│                                                             │
│  • Vector Search & RAG           →  Customer Support Bots   │
│  • Async Processing              →  Fraud Detection Systems │
│  • Model Optimization            →  Medical Image Analysis  │
│  • Multi-Tenant Architecture     →  SaaS AI Platforms       │
│  • Distributed Serving           →  Real-time Recommenders  │
│  • MLOps & Monitoring            →  Any Production AI App   │
└─────────────────────────────────────────────────────────────┘

Remember: The chatbot is just the vehicle. The destination is becoming a production AI engineer.

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🌟 One Chatbot. Six Layers. Production Ready. 🌟

Layer Evolution:

v1.0 (API) → v2.0 (RAG) → v3.0 (PostgreSQL) → 
v4.0 (Automation) → v5.0 (MLOps) → v6.0 (Scale)

Universal Engineering Principles for All AI Applications

Learning in Public, Building for Production

Made with ❤️, lots of ☕, and a healthy dose of 🔥

GitHub followers Twitter Follow

⬆ Back to Top

Build it layer by layer, ship it to production 🚀

About

MyStack is a comprehensive, documented journey through all the necessary components to build production-grade AI-powered applications.

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages