PassedAI 🔍

Multi-agent knowledge decay auditor — Built for the Elasticsearch Agent Builder Hackathon

---

🎯 The Problem

Enterprise knowledge bases decay silently. Documentation becomes outdated, contradictory, or concentrated in the hands of a few experts — creating operational risk that's invisible until something breaks.

PassedAI detects four types of knowledge decay automatically:

Decay Type	What It Catches	Business Impact
🕐 Stale Docs	Documents not updated in 6-24+ months	Employees following outdated procedures
⚔️ Contradictions	Two docs giving opposite guidance on the same topic	Confusion, compliance risk
❓ Knowledge Gaps	Repeated employee questions with no matching documentation	Tribal knowledge, onboarding friction
👤 Expert Risk	Single owners of critical documentation	Bus factor = 1, knowledge silos

---

🏗️ Architecture

flowchart TB
    subgraph UI["🖥️ Demo UI (Flask :5050)"]
        Dashboard["Real-time Dashboard"]
    end

    subgraph Orchestrator["🐍 Python Orchestrator"]
        RunAudit["run_audit.py"]
        A2AClient["a2a_client.py"]
    end

    subgraph AgentBuilder["⚡ Elastic Agent Builder"]
        subgraph Detect["PassedAI-Detect"]
            D1["staleness_scorer"]
            D2["gap_detector"]
            D3["contradiction_finder"]
            D4["expert_risk_finder"]
        end
        subgraph Act["PassedAI-Act"]
            A1["notify_owner"]
            A2["create_gap_task"]
        end
    end

    subgraph ES["🔍 Elasticsearch Serverless"]
        Docs["passed_documents<br/>200 docs + ELSER v2"]
        Questions["passed_questions<br/>60 questions + ELSER v2"]
        AuditLog["passed_audit_log"]
        Notifications["passed_notifications"]
    end

    Reports["📄 Audit Reports<br/>data/reports/*.md"]

    Dashboard -->|REST API| ES
    RunAudit -->|A2A| Detect
    Detect -->|ESQL| ES
    Detect -->|Findings| RunAudit
    RunAudit -->|A2A| Act
    Act -->|Actions| Notifications
    RunAudit --> Reports

Loading

Data Flow

sequenceDiagram
    participant O as Orchestrator
    participant D as Detect Agent
    participant A as Act Agent
    participant ES as Elasticsearch

    O->>D: A2A message/send audit prompt
    D->>ES: ESQL staleness_scorer
    D->>ES: ESQL gap_detector
    D->>ES: ESQL contradiction_finder
    D->>ES: ESQL expert_risk_finder
    D-->>O: Structured findings STALE/GAP/CONTRADICTION/EXPERT_RISK
    
    O->>A: A2A message/send (findings)
    A-->>O: ACTION lines + ACTION_SUMMARY
    
    O->>O: Write markdown report

Loading

---

✨ Elastic Features Used

Feature	Implementation	Purpose
ES|QL Tools	4 custom queries in Agent Builder	Detect stale docs, gaps, contradictions, expert risk
A2A Protocol	JSON-RPC 2.0 message/send	Orchestrator ↔ Agent communication
ELSER v2	semantic_text field type	Semantic search for contradiction detection
Agent Builder	2 agents with custom system prompts	Detect + Act multi-agent pattern
Serverless	Elastic Cloud Serverless	Zero-ops deployment

---

📊 Demo Results

Sample audit output from synthetic data:

╭────────────────────────────────────────────╮
│ PassedAI Audit                             │
│ Run ID: 0ec4c448  Window: 90d  Top-N: 10   │
╰────────────────────────────────────────────╯

         Detect Results           
                                   
  Category                  Count  
 ───────────────────────────────── 
  Stale docs (score ≥ 70)      20  
  Knowledge gaps                1  
  Contradiction pairs           2  
  Expert risk owners            8  

Dashboard Screenshot: The Flask UI at localhost:5050 shows real-time stats pulled from Elasticsearch.

---

📁 Repository Structure

PassedAI/
├── config/
│   ├── agent_prompts/           # System prompts for both agents
│   │   ├── detect_agent.md
│   │   └── act_agent.md
│   ├── esql/                    # ES|QL tool queries
│   │   ├── staleness_scorer.esql
│   │   ├── gap_detector.esql
│   │   ├── contradiction_finder.esql
│   │   └── expert_risk_finder.esql
│   └── index_mappings/          # Elasticsearch index schemas
├── data/
│   ├── generated/               # Synthetic test data
│   └── reports/                 # Audit output (markdown)
├── docs/
│   └── kibana_setup.md          # Step-by-step Kibana configuration
├── orchestrator/
│   ├── a2a_client.py            # A2A protocol client
│   └── run_audit.py             # Main orchestrator script
├── scripts/
│   ├── generate_synthetic_data.py
│   └── ingest.py
├── ui/
│   ├── app.py                   # Flask dashboard
│   └── templates/index.html
├── .env.example
├── LICENSE (MIT)
├── README.md
└── requirements.txt

---

🚀 Quickstart

Prerequisites

• Python 3.11+
• Elastic Cloud Serverless (free trial)

1. Install

git clone https://github.com/N-45div/PassedAI.git
cd PassedAI
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

2. Configure .env

Variable	Source
ELASTIC_CLOUD_ID	Elastic Cloud → Project → Manage → Cloud ID
ELASTIC_API_KEY	Elastic Cloud → API Keys → Create
KIBANA_BASE_URL	Derived from Cloud ID (see docs)
KIBANA_API_KEY	Kibana → Stack Management → API Keys
PASSED_DETECT_AGENT_ID	After creating agent in Kibana
PASSED_ACT_AGENT_ID	After creating agent in Kibana

3. Generate & Ingest Data

python scripts/generate_synthetic_data.py
python scripts/ingest.py --reset

4. Configure Kibana

Follow docs/kibana_setup.md:

1. Verify ELSER v2 is running
2. Create 4 ES|QL tools
3. Create PassedAI-Detect and PassedAI-Act agents
4. Copy agent IDs to .env

5. Run Audit

python orchestrator/run_audit.py --audit-window-days 90 --top-n-actions 10

6. View Dashboard

python ui/app.py
# Open http://localhost:5050

---

🔧 How It Works

Detect Phase

The PassedAI-Detect agent runs 4 ES|QL queries in sequence:

1. staleness_scorer — Finds docs with staleness_score >= 70
2. gap_detector — Finds unanswered questions (knowledge gaps)
3. contradiction_finder — Semantic search for conflicting docs on same topic
4. expert_risk_finder — Aggregates doc ownership to find concentration risk

Act Phase

The PassedAI-Act agent processes findings and outputs structured action logs:

ACTION | notify_owner | doc_id=abc123 | owner=alice@acme.io | issue=stale | score=100
ACTION | notify_owner | topic=data-retention | issue=contradiction | docs=doc1,doc2

Orchestrator

The Python orchestrator (run_audit.py):

1. Sends audit prompt to Detect agent via A2A
2. Parses structured findings
3. Sends findings to Act agent via A2A
4. Writes markdown report to data/reports/

---

📝 Challenges & Learnings

1. A2A Protocol Discovery — Elastic Agent Builder uses message/send method with kind: "text" parts (not type). Required debugging the JSON-RPC format.

2. Serverless ES Restrictions — Index mappings can't include number_of_shards / number_of_replicas settings. Had to remove these for Serverless compatibility.

3. ELSER Inference Timeouts — Bulk ingestion with semantic_text fields requires longer timeouts (120s) and smaller chunk sizes (10 docs) due to inference processing time.

---

📜 License

MIT License — see LICENSE

---

🙏 Acknowledgments

Built with:

• Elasticsearch Agent Builder
• ELSER v2
• A2A Protocol

---

Built for the Elasticsearch Agent Builder Hackathon 🏆