BillSense — AI Billing Agent for Energy Utilities

BillSense is a production-grade AI assistant that transforms electricity bill images into structured, personalized, and grounded explanations using Vision + Structured Data + RAG.

Built for the NPTryHard Hackathon — Vision-to-RAG Prompt Builder for Energy Billing challenge.

---

Overview

BillSense simulates an enterprise AI Billing Agent for a Greek energy utility (DEH/PPC). Customers upload a photo of their electricity bill and receive an accurate, citation-backed analysis — powered by multimodal AI, a simulated Data Warehouse, and Retrieval-Augmented Generation.

Key Capabilities

• Bill Image Extraction — OCR + LLM structured extraction from bill photos/PDFs
• Customer Matching — Automatic DWH lookup with fallback strategy and ambiguity handling
• RAG-Grounded Answers — Multi-query retrieval, reranking, and citation-based responses
• Anti-Hallucination Guards — Multi-layer verification (judge + post-generation sanitization)
• Personalized Analysis — Consumption comparison, anomaly detection, savings recommendations
• Bilingual UI — Full Greek & English support with voice input

---

Architecture

  Bill Image (PNG/JPG/PDF)
          |
          v
  +-----------------------+
  |  Step 1: Vision &     |  Gemini Vision OCR -> LLM Structured Extraction
  |  Data Extraction      |  -> Confidence-scored merge
  +----------+------------+
             | Extracted identifiers
             v
  +-----------------------+
  |  Step 2: Customer     |  supply_number -> customer_id -> contract_id
  |  Matching (DWH)       |  1 match / multiple / none handling
  +----------+------------+
             | Customer context
             v
  +-----------------------+
  |  Step 3: RAG Prompt   |  Multi-query expansion (3 sub-queries)
  |  Package Creation     |  -> Vector retrieval (pgvector)
  |                       |  -> Reranking (Cohere / Gemini fallback)
  |                       |  -> Metadata filters + system instructions
  +----------+------------+
             | Retrieved knowledge + context
             v
  +-----------------------+
  |  Step 4: Grounded     |  Personalized answer generation
  |  Answer Generation    |  -> LLM-as-Judge quality scoring
  |                       |  -> Post-generation hallucination verification
  |                       |  -> Confidence calibration (cosine+reranker+judge)
  +-----------------------+

For full technical details with Mermaid diagrams, see ARCHITECTURE.md

---

Tech Stack

Layer	Technology
Frontend	React 19, Next.js 16 (App Router), Tailwind CSS 4, Framer Motion
AI/LLM	Google Gemini (Vision, Embeddings, Structured Output, Judge)
RAG	LangChain, PGVectorStore (pgvector), Cohere Rerank v3.5
Database	Neon PostgreSQL (serverless), Prisma ORM
Voice	OpenAI Whisper (Greek transcription)
Evaluation	LangSmith (5 evaluators: factual overlap, confidence, retrieval, groundedness, language)
i18n	next-intl (Greek + English)
Deployment	Vercel (serverless functions)

For detailed tech stack mapping to challenge requirements, see docs/TECH_STACK.md

---

Project Structure

billsense/
  src/
    app/                        # Next.js App Router pages & API routes
      api/
        chat/                   # Streaming chat with RAG orchestration
        extract/                # OCR + LLM extraction pipeline
        transcribe/             # Voice-to-text (Whisper)
        bills/                  # Bill CRUD operations
        conversations/          # Chat persistence
      chat/                     # Chat interface pages
      bills/                    # Bill library & comparison pages
    features/
      extraction/               # Stage 1: OCR + LLM extraction + merge
      customer/                 # Stage 2: DWH customer matching
      retrieval/                # Stage 3: RAG pipeline (multi-query, rerank, prompts)
      orchestrator/             # Stage 4: End-to-end pipeline orchestration
    components/                 # React UI components (50+)
    hooks/                      # Custom hooks (browser ID, voice input)
    lib/
      db/                       # Database ingestion (DWH + Knowledge Base)
      eval/                     # RAG evaluation with LangSmith
    i18n/                       # Internationalization config
  prisma/
    schema.prisma               # Database schema (9 DWH tables + app tables)

---

Quick Start

Prerequisites

• Node.js 20+
• Neon PostgreSQL database with pgvector extension
• API keys: Gemini, Cohere (optional), OpenAI (optional, for voice)

Setup

cd billsense

# Install dependencies
npm install

# Configure environment
cp .env.example .env
# Edit .env with your API keys and database URL

# Push schema & seed DWH + Knowledge Base
npm run db:seed

# Start development server
npm run dev

Environment Variables

# Required
DATABASE_URL=postgresql://...        # Neon PostgreSQL connection string
GEMINI_API_KEY=...                   # Google Gemini API key

# Optional (enhances quality)
COHERE_API_KEY=...                   # Cohere reranking (fallback: Gemini judge)
OPENAI_API_KEY=...                   # Whisper voice transcription
LANGSMITH_API_KEY=...                # RAG evaluation tracking

For detailed seeding instructions, see docs/SEEDING.md

---

Evaluation

BillSense includes a LangSmith-based evaluation framework with 5 custom evaluators:

cd billsense && npm run eval

Evaluator	What it measures
factualOverlap	Accuracy of answer vs. reference
confidenceCheck	Confidence score calibration
hasRetrieval	Presence of source citations
groundedness	Answer grounded in retrieved documents
languageCheck	Greek language quality

---

Documentation

Document	Description
ARCHITECTURE.md	Full system architecture with Mermaid diagrams
RETRIEVAL_STRATEGY.md	Multi-query RAG pipeline, reranking, confidence calibration
SAFETY.md	Anti-hallucination guards, 5-layer defense-in-depth
TECH_STACK.md	Tech stack mapped to challenge requirements
SEEDING.md	Database seeding guide
DEI_BILL_OCR_SYSTEM_PROMPT_v2.md	OCR extraction prompt reference

---

Team

NP-TryHards