Repository navigation
Releases
v0.4.0 — SRE troubleshooting with GraphRAG
Highlights
PolicyDesk now has a second domain for SRE incident troubleshooting, next to the existing policy assistant. The policy domain is unchanged and remains the default.
With KB_DOMAIN=oncall, PolicyDesk combines a service catalog in Neo4j, runbooks and postmortems in Markdown, and GraphRAG retrieval to answer questions like:
- If product-catalog is down, which services are affected?
- checkout latency is up and payment errors are rising. What is the likely root cause?
- Who owns valkey-cart and where do we escalate?
- Has this Kafka consumer lag happened before? How was it fixed?
What's new
Knowledge graph
- Service catalog import:
service_catalog.yamlandalerts.yamlare validated and loaded into Neo4j without an LLM, covering services, datastores, teams, dependencies, owners, escalation paths, and alerts linked to runbook sections. - Postmortem extraction: Qwen extracts incidents, affected services, root causes, symptoms, and mitigations with a strict schema. Extracted names are aligned to the catalog, including Chinese aliases, and symptoms to a controlled vocabulary. Results are cached in Redis.
- Safe provenance: every graph fact records the documents and chunks that support it. Deleting a postmortem removes only facts that no other document supports. A failed extraction changes nothing, and a failed graph write rolls back.
Retrieval
- Three modes:
hybrid(the existing chain, used as the baseline),graph, andhybrid+graph. - Deterministic entity linking on the original question and fixed Cypher templates; the LLM never writes Cypher.
- Root-cause ranking based on shared downstream dependencies, distance, and incident history.
- Graph facts keep reserved slots after reranking, so the reranker cannot drop them.
- The catalog is also rendered as text, so the hybrid baseline sees the same facts and the comparison is fair.
Interface
- Web demo: mode switch, side-by-side comparison (hybrid vs hybrid + graph), separate document evidence and graph reasoning paths, catalog import, Markdown upload, and alert updates without re-uploading the catalog.
- CLI:
import-catalog, Markdownadd, and--mode. - Metrics: retrieval, first-token, generation, and total time, plus LLM calls and estimated tokens, for every answer.
- Structured output: each answer ends with a JSON block (
root_cause_services,affected_services,owner_team,uncertain) for evaluation.
Demo corpus
examples/oncall_demo/ contains a 16-component shop catalog, 5 alerts, 5 runbooks, 5 postmortems, and 16 test questions with expected answers.
Upgrade
pip install -r requirements.txt
docker compose --profile oncall up -d --waitv0.3.0 — Upload your own PDFs, ask in Chinese or English
PolicyDesk now builds each company's knowledge base from the policy PDFs it uploads. Tables and figures are parsed, employees can ask in Chinese or English, and the web app has been redesigned around two views: Ask and Documents.
Highlights
- Upload your own PDFs. PDFs are parsed with MinerU 4.x; figures are described by Qwen-VL (
qwen3-vl-plus). New documents are added to the existing index, with no full rebuild. - Figures that fail are skipped, not fatal. The rest of the PDF is still added, and the app lists each skipped figure's page, caption, and reason.
- Add, list, and delete documents from the web app or the CLI. Deletion removes a document from Chroma, MongoDB, and Redis together. The same file cannot be added twice.
- Ask in Chinese or English. Query rewriting and answers follow the question's language, and keyword search now matches Chinese text.
- Redesigned web app. Source cards show file, page, and type (text, table, figure) before the streamed answer. The Documents view shows upload progress and a status table with confirmed deletion. Chinese/English interface, light and dark themes, phone layout.
⚠️ Upgrade notes
- Use a new knowledge base. The default
KB_NAMEis nowpolicydesk_uploads_v1. Set a new, unusedKB_NAMEin.env; older namespaces still contain the sample policies. - Sample policies are no longer indexed. Files in
data/are kept only as test fixtures. - Install MinerU separately. Install MinerU 4.x in its own environment and set
MINERU_COMMANDto the full path of itsmineruexecutable, for example/opt/anaconda3/envs/mineru/bin/mineru. - CLI changes.
preview,build, and--categorywere removed. Useadd,delete,list,ask, andchat. - New optional settings:
MINERU_PYTHON,MINERU_TIER,VL_MODEL.
Getting started
git pull
python -m pip install -r requirements.txt
docker compose up -d --wait
python -m streamlit run app.pyPolicyDesk v0.2.0 — Scoped Advanced RAG
PolicyDesk v0.2.0 introduces an advanced retrieval pipeline with business metadata, optional policy scopes, hybrid search, and local relevance reranking.
Employees can search the full policy library or select a specific category, with inspectable policy excerpts supporting each answer.
Added
- Business metadata: validated policy IDs, titles, versions, owners, document types, languages, and categories inherited by indexed nodes.
- Scoped retrieval: category and document-type filters applied consistently to Chroma vector search and the BM25 corpus.
- Query rewriting: one English retrieval query with numeric-preservation checks and fallback to the original question.
- Hybrid search: semantic retrieval and BM25 combined through reciprocal rank fusion.
- Local reranking: BGE selects up to four relevant excerpts before answer generation.
- Scope controls: category selection in the Streamlit interface and CLI, with separate CLI conversation histories for each scope.
- Source details: policy IDs, versions, categories, and reranker scores displayed alongside retrieved excerpts.
Fixed and improved
- Aligned retrieval helpers and category-aware service interfaces.
- Reused cached models across scope-specific web query engines.
- Cleared previous web answers when changing search scope.
- Rejected empty model output instead of displaying success or recording an empty chat answer.
- Restricted Chroma collection creation to index-building operations.
- Expanded regression tests and updated dependency, configuration, and setup documentation.
Upgrade notes
Install the updated dependencies from requirements.txt.
Existing indexes do not automatically receive business metadata. Choose a new KB_NAME, run python main.py build, and restart Streamlit. Preserve existing database and Docker volume settings to retain previous indexes and conversations.
Current limitations
PolicyDesk remains a simplified local application. Category filters control search scope and do not provide access control. Policy updates require a new index namespace, and version metadata does not automatically resolve superseded policies.
Qwen inference requires a valid DashScope API key and available account quota. Reranker scores indicate relative relevance, not answer confidence.
PolicyDesk v0.1.0 — Basic RAG Policy Assistant
Overview
The first release of PolicyDesk, a lightweight company policy assistant built with LlamaIndex.
Features
- Policy question answering powered by Qwen Plus.
- Local HuggingFace embeddings for document retrieval.
- Persistent storage using MongoDB, Chroma, and Redis.
- Streamlit interface with streamed answers and retrieved policy sources.
- Named conversation sessions with saved history.
Getting Started
Follow the README to install dependencies, configure your DashScope API key, start the database services, and build the policy index.
Current Scope
This release focuses on a basic RAG workflow. Pre-retrieval processing, hybrid retrieval, and reranking are planned for future releases.
Answers should be checked against the retrieved policy sources.