AI-Driven Ticket Handling System

Project Overview

Intelligent auto-handling of support tickets with confidence-based Human-in-the-Loop (HITL) system. The system classifies incoming tickets, identifies known issues using vector similarity search, automatically resolves high-confidence tickets, and escalates uncertain cases to human agents.

Progress Status

✅ Day 1 Complete - Environment & Skeleton Setup

What's Working:

• ✅ Monorepo structure created (/backend, /frontend)
• ✅ FastAPI skeleton with SQLite database
• ✅ POST /tickets endpoint (accepts title + description, returns JSON)
• ✅ GET /tickets endpoint (retrieves all tickets)
• ✅ Health check endpoint
• ✅ Database models defined (Ticket, Resolution, HumanQueue, AuditLog, HistoricalTicket)
• ✅ Pydantic schemas for validation
• ✅ CORS middleware configured for React frontend

Test Results:

# Server running on http://localhost:8000
curl -X POST "http://localhost:8000/tickets" \
  -H "Content-Type: application/json" \
  -d '{"title": "Cannot login with SSO", "description": "Okta integration failing, getting 401 error", "submitter_email": "test@company.com"}'

# Response:
{
  "id": "daf7ee69-9355-4e20-b420-7476af54813e",
  "title": "Cannot login with SSO",
  "description": "Okta integration failing, getting 401 error every time I try to login",
  "category": null,
  "priority": "MEDIUM",
  "status": "NEW",
  "ai_confidence": null,
  "submitter_email": "test@company.com",
  "created_at": "2026-03-24T17:30:45.941858",
  "updated_at": "2026-03-24T17:30:45.941862"
}

✅ Day 2 Complete - Agent 1: Gemini Classification

What's Working:

• ✅ Agent 1: Ticket Classifier using Gemini 2.5 Flash (with mock fallback)
• ✅ Automatic classification into 6 categories (AUTH, BILLING, INFRA, UI_BUG, API_ERROR, OTHER)
• ✅ Confidence scoring (0-100%)
• ✅ Integration with ticket creation workflow
• ✅ Audit trail for all AI decisions
• ✅ Status progression: NEW → CLASSIFIED
• ✅ Direct /classify endpoint for testing
• ✅ Enhanced /tickets endpoint with classification

Test Results:

# Agent 1 Classification Examples:
"Cannot login with SSO" → AUTH (85% confidence)
"Invoice billing issue" → BILLING (82% confidence)
"Server deployment failed" → INFRA (88% confidence)
"API endpoint 500 error" → INFRA (88% confidence)

# Full Workflow Test:
POST /tickets → Creates ticket + Classifies + Updates status + Audit logs
Status: NEW → CLASSIFIED in single API call

✅ Day 3 Complete - Agent 2: Vector Similarity Search

What's Working:

• ✅ Agent 2: Vector Similarity Searcher using Gemini Embeddings (with mock fallback)
• ✅ 8 realistic historical tickets seeded across 6 categories
• ✅ Pure Python cosine similarity calculation (no external dependencies)
• ✅ Vector embedding generation and storage in SQLite as JSON
• ✅ Top-K similar ticket retrieval with configurable similarity thresholds
• ✅ Comprehensive API endpoints for similarity operations
• ✅ Statistics and monitoring for embedding coverage

New Endpoints:

# Seed historical tickets (run once)
POST /admin/seed-historical-tickets

# Generate embeddings for historical tickets
POST /admin/embed-historical-tickets

# Search for similar tickets
POST /similarity-search

# Get similarity system statistics
GET /similarity/stats

Test Results:

# Example similarity search:
Input: "Cannot authenticate with corporate SSO"
Results:
1. [AUTH] SSO login fails with 401 error (similarity: 0.842)
2. [AUTH] Password reset emails not being sent (similarity: 0.721)
3. [AUTH] Multi-factor authentication not working (similarity: 0.687)

# System Statistics:
✅ Total historical tickets: 8
✅ Embedded tickets: 8
✅ Embedding coverage: 100%
✅ Categories: AUTH(2), INFRA(2), API_ERROR(2), UI_BUG(1), BILLING(1)

✅ Day 4 Complete - Agent 3: Confidence Scoring + LangGraph Pipeline

What's Working:

• ✅ Agent 3: Confidence Scorer with weighted formula similarity × 0.6 + category_match × 0.4
• ✅ LangGraph pipeline orchestrating Agent 1 → Agent 2 → Agent 3 → Router
• ✅ Confidence-based routing: ≥75% auto-resolve, 50-75% human review, <50% escalate
• ✅ Complete ticket processing with pipeline integration
• ✅ Human queue management for medium-confidence tickets
• ✅ Comprehensive audit logging for all pipeline steps
• ✅ Pipeline statistics and monitoring endpoints

New Endpoints:

# Test complete pipeline without creating ticket
POST /pipeline/test

# Get comprehensive pipeline statistics
GET /pipeline/stats

# Enhanced ticket creation (now uses full pipeline)
POST /tickets  # Updated to use Agent 1+2+3 pipeline

Pipeline Flow:

Incoming Ticket
     ↓
Agent 1: Classify → AUTH (85% confidence)
     ↓
Agent 2: Search → 3 similar tickets (0.84, 0.72, 0.68 similarity)
     ↓
Agent 3: Calculate → Final confidence = 0.78 (formula-based)
     ↓
Router: Decision → AUTO_RESOLVE (confidence ≥ 75%)

Routing Logic:

• ≥75% confidence → AUTO_RESOLVED status (no human needed)
• 50-75% confidence → PENDING_REVIEW status + added to human queue
• <50% confidence → ESCALATED status (requires investigation)

🔄 Currently Working - Day 5: Agent 4&5 (Resolution + Escalation)

Architecture

Tech Stack

• Backend: Python FastAPI + SQLite + Uvicorn
• AI Models: Google Gemini (Flash for speed, Pro for quality)
• Vector DB: Qdrant (local Docker - pending setup)
• Agent Orchestration: LangGraph (to be implemented)
• Frontend: React + TypeScript + Tailwind CSS (pending)

Current Database Schema

# Core Models
Ticket: id, title, description, category, priority, status, ai_confidence, submitter_email, timestamps
AuditLog: id, ticket_id, event_type, actor, details, timestamp
HumanQueue: id, ticket_id, ai_recommendation, confidence_score, status, reviewed_by, human_notes
Resolution: id, ticket_id, ai_recommendation, applied_solution, is_auto_resolved, approved_by
HistoricalTicket: id, title, description, category, solution, embedding, usage_count

Confidence-Based Routing (To Be Implemented)

Incoming Ticket
    ↓
Agent 1: Classify (Gemini Flash) → category + confidence
    ↓
Agent 2: Search similar tickets (Qdrant) → top-3 matches
    ↓
Agent 3: Calculate confidence → similarity × 0.6 + category_match × 0.4
    ↓
Route Decision:
├─ Confidence ≥ 75% → Auto-resolve (Agent 4: Gemini Pro)
└─ Confidence < 75% → Escalate (Agent 5: Human Queue)

Running the Project

Backend

cd backend
pip install fastapi uvicorn sqlalchemy python-dotenv
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

Test Endpoints

# Health check
curl http://localhost:8000/health

# Create ticket
curl -X POST http://localhost:8000/tickets \
  -H "Content-Type: application/json" \
  -d '{"title": "Test issue", "description": "Sample description"}'

# Get all tickets
curl http://localhost:8000/tickets

Environment Setup

1. Copy .env.example to .env
2. Add your Google AI Studio API key:

   GOOGLE_API_KEY=your_gemini_api_key_here
   

Next Milestones

• Day 2: Agent 1 - Gemini classification with confidence scoring
• Day 3: Agent 2 - Vector embeddings + Qdrant similarity search
• Day 4: Agent 3 - Confidence scoring + LangGraph pipeline integration
• Day 5: Agent 4&5 - Auto-resolution + escalation to human queue
• Day 6-7: React dashboard + end-to-end demo

Current Status: ✅ Day 4 Complete, 🔄 Day 5 In Progress

Next Steps for Agent 4&5:

• Agent 4: Resolution Generator using Gemini Pro (high-quality solutions)
• Agent 5: Escalation Handler (structured human queue management)
• Integration with pipeline for auto-resolution path
• Enhanced human dashboard for approval workflow
• End-to-end testing of complete HITL system