AI_Feature

AI Feature Architecture

Overview

The project ships an AI assistant that lets employees ask natural-language questions about their project-management data ("what should I prioritize?", "which of my tasks are blocked?") and get grounded, role-appropriate answers. It is reachable from two surfaces — the Telegram bot and a REST chat endpoint in the web app — both backed by the same BotAgentService.

The assistant is a Retrieval-Augmented Generation (RAG) system: instead of stuffing the whole project state into the prompt, it embeds the question, retrieves only the most relevant chunks from an Oracle 26ai vector store, and asks the LLM to answer from that context alone. The retrieval/embedding internals are documented separately in RAG.md; this document covers the end-to-end AI feature and how the pieces fit together.

   ┌─────────────┐        ┌─────────────────┐
   │ Telegram    │        │ Web app /chat   │
   │ /ask  /llm  │        │ REST endpoint   │
   └──────┬──────┘        └────────┬────────┘
          │  Employee (role-scoped)│
          └───────────┬────────────┘
                      ▼
            ┌───────────────────┐
            │  BotAgentService  │   processQuery(employee, query)
            └─────────┬─────────┘
            ┌─────────┴───────────────────────────┐
            ▼ (1) retrieve                        ▼ (3) generate
   ┌───────────────────┐                   ┌───────────────────┐
   │  RagRetriever     │                   │ OpenRouterService │
   │  (role-scoped     │                   │  chat(system,     │
   │   vector search)  │                   │       user)       │
   └────────┬──────────┘                   └────────┬──────────┘
            │ embed + VECTOR_DISTANCE               │ HTTPS
            ▼                                       ▼
   ┌───────────────────┐                   ┌──────────────────┐
   │ Oracle 26ai       │                   │  OpenRouter API  │
   │ RAG_CHUNK + index │                   │  (configurable   │
   │ (in-DB embeddings)│                   │   LLM model)     │
   └───────────────────┘                   └──────────────────┘

Two LLM-backed commands

The assistant exposes two distinct behaviors (see Bot-Architecture.md for the full command table):

Command	Path	Context	Use
/ask <question>	BotAgentService.processQuery	RAG — retrieved, role-scoped project chunks	Grounded answers about the user's own data
/llm <prompt>	OpenRouterService.generateText	none	Raw passthrough to the LLM for general questions

/ask is the RAG feature. /llm is a deliberately context-free escape hatch.

Request flow (/ask)

BotAgentService.processQuery(Employee employee, String query):

1. Determine role. isManager = role == "manager" || role == "admin". This single boolean drives both retrieval scope and the system prompt.

2. Retrieve (RagRetriever.search(query, topK, employee, isManager)):

   • The question is embedded in-database with VECTOR_EMBEDDING using the same ALL_MINILM_L12_V2 model that indexed the data.
   • A cosine VECTOR_DISTANCE search returns the top-K chunks (rag.retrieval.top-k, default 8).
   • Access control lives in the SQL WHERE clause: managers see all chunks; developers see organizational sprint/project chunks plus only the task chunks they're assigned to. There is no way for the LLM to receive a chunk the caller isn't entitled to.
   • If retrieval throws, the user gets a graceful "couldn't search the project data right now" message rather than an error.

3. Assemble the prompt. Retrieved chunk texts are joined with --- separators into a === RETRIEVED CONTEXT === block, followed by === QUESTION === and the raw query. If nothing was retrieved, the context is an explicit "(no relevant project data was found)" sentinel so the model doesn't hallucinate.

4. Generate (OpenRouterService.chat(systemPrompt, userMessage)): a system+user message pair is POSTed to OpenRouter. LLM failures also degrade gracefully to a friendly fallback string.

System prompt

buildSystemPrompt constructs the instructions per request. Key directives:

• Identity & role — the user's name and role, and (for developers) an explicit note that the context only includes their own tasks plus organizational sprints/projects, with a "do not invent data about other employees" guard.
• Grounding — "Answer ONLY based on the RETRIEVED CONTEXT section. If it doesn't contain enough information, say so plainly. Do NOT guess from prior knowledge." This is the core anti-hallucination rule and the mitigation for the absence of a re-ranker.
• Citation anchor — chunks are prefixed with [TASK #id] / [SPRINT #id] / [PROJECT #id]; the prompt tells the model to refer to those IDs.
• Formatting — plain text only (no Markdown), conversational tone, status and priority emojis (🔄 📋 ✅ 🚫, 🔴 🟡 🟢), a compact per-task layout, and "answer in the user's language" — tuned for readability on a phone.

Components

Component	Responsibility
BotAgentService	Orchestrates retrieve → assemble → generate; builds the system prompt; handles failures gracefully
RagRetriever	Embeds the query and runs the role-scoped top-K vector search (see RAG.md)
EmbeddingService	Single wrapper over VECTOR_EMBEDDING(model USING ? AS DATA); used by both indexing and retrieval
OpenRouterService	HTTP client for the OpenRouter chat/completions API (chat() for RAG, generateText() for /llm)
RAG indexing layer	RagDirtyEnqueuer, RagIndexWorker, RagIndexer, RagBackfillRunner — keep RAG_CHUNK in sync with entity writes (see RAG.md)

The LLM — OpenRouter

The assistant talks to a hosted LLM through OpenRouter, not a Claude/OpenAI SDK directly. OpenRouterService builds a JSON messages array (system + user) and POSTs it with Apache HttpClient5 to https://openrouter.ai/api/v1/chat/completions. The model is configurable; the default is google/gemini-2.5-flash-lite.

Property	Env	Default
llm.api.key	OPEN_ROUTER_API_KEY	— (required)
llm.api.url	—	https://openrouter.ai/api/v1/chat/completions
llm.model	LLM_MODEL	google/gemini-2.5-flash-lite

The chat model is intentionally pluggable and separate from the embedding model. The embedding side runs in-database (Oracle ONNX, no API); only the final generation step is an external call.

The embedding + retrieval side

Covered in full in RAG.md. In short: every task/sprint/project is embedded into a VECTOR(384, FLOAT32) column in RAG_CHUNK by a loaded ALL_MINILM_L12_V2 ONNX model, kept fresh asynchronously via the RAG_DIRTY queue and RagIndexWorker, and searched with cosine VECTOR_DISTANCE over an Oracle AI Vector Search index.

Why RAG instead of a full context dump

The earlier assistant dumped the user's entire visible dataset into every prompt — a developer's full task list, or, for a manager, every employee, task, sprint, and project. That fails predictably as the organization grows: the prompt size scales linearly, it bursts the LLM context window and spend limit within a few sprints, and answer quality degrades because the model can't separate relevant data from noise. RAG sends only the chunks semantically close to the actual question, which keeps prompts small, cheap, and on-topic, and makes the system usable at organizational scale.

Configuration summary

Property	Default	Purpose
llm.model / LLM_MODEL	google/gemini-2.5-flash-lite	Generation model
llm.api.key / OPEN_ROUTER_API_KEY	—	OpenRouter credential
rag.retrieval.top-k	8	Chunks fed to the LLM per question
rag.embedding.model	ALL_MINILM_L12_V2	In-DB embedding model name
(indexing knobs)	see RAG.md	Backfill, worker cadence, batch size, retry cap

No new secrets are introduced by the AI feature beyond the pre-existing OPEN_ROUTER_API_KEY: the embedding model lives inside the database.

Failure handling

The feature degrades gracefully at every external boundary:

• Retrieval failure (DB/embedding) → "Sorry, I couldn't search the project data right now."
• Empty retrieval → the model is told no data was found, so it says it doesn't know rather than inventing an answer.
• LLM failure (OpenRouter) → "Sorry, I couldn't process your request right now."
• Indexing failures are isolated to the background worker and never block a chat or an entity write; they retry from the queue (see RAG.md).

Related documents

• RAG.md — the embedding, indexing, and retrieval internals.
• Bot-Architecture.md — the Telegram bot surface, commands, auth, and data model.