Skip to content

Releases

v0.4.0 — SRE troubleshooting with GraphRAG

Choose a tag to compare

@Chuck0816 Chuck0816 released this 18 Nov 04:13
55b01db

Highlights

PolicyDesk now has a second domain for SRE incident troubleshooting, next to the existing policy assistant. The policy domain is unchanged and remains the default.

With KB_DOMAIN=oncall, PolicyDesk combines a service catalog in Neo4j, runbooks and postmortems in Markdown, and GraphRAG retrieval to answer questions like:

  • If product-catalog is down, which services are affected?
  • checkout latency is up and payment errors are rising. What is the likely root cause?
  • Who owns valkey-cart and where do we escalate?
  • Has this Kafka consumer lag happened before? How was it fixed?

What's new

Knowledge graph

  • Service catalog import: service_catalog.yaml and alerts.yaml are validated and loaded into Neo4j without an LLM, covering services, datastores, teams, dependencies, owners, escalation paths, and alerts linked to runbook sections.
  • Postmortem extraction: Qwen extracts incidents, affected services, root causes, symptoms, and mitigations with a strict schema. Extracted names are aligned to the catalog, including Chinese aliases, and symptoms to a controlled vocabulary. Results are cached in Redis.
  • Safe provenance: every graph fact records the documents and chunks that support it. Deleting a postmortem removes only facts that no other document supports. A failed extraction changes nothing, and a failed graph write rolls back.

Retrieval

  • Three modes: hybrid (the existing chain, used as the baseline), graph, and hybrid+graph.
  • Deterministic entity linking on the original question and fixed Cypher templates; the LLM never writes Cypher.
  • Root-cause ranking based on shared downstream dependencies, distance, and incident history.
  • Graph facts keep reserved slots after reranking, so the reranker cannot drop them.
  • The catalog is also rendered as text, so the hybrid baseline sees the same facts and the comparison is fair.

Interface

  • Web demo: mode switch, side-by-side comparison (hybrid vs hybrid + graph), separate document evidence and graph reasoning paths, catalog import, Markdown upload, and alert updates without re-uploading the catalog.
  • CLI: import-catalog, Markdown add, and --mode.
  • Metrics: retrieval, first-token, generation, and total time, plus LLM calls and estimated tokens, for every answer.
  • Structured output: each answer ends with a JSON block (root_cause_services, affected_services, owner_team, uncertain) for evaluation.

Demo corpus

examples/oncall_demo/ contains a 16-component shop catalog, 5 alerts, 5 runbooks, 5 postmortems, and 16 test questions with expected answers.

Upgrade

pip install -r requirements.txt
docker compose --profile oncall up -d --wait

v0.3.0 — Upload your own PDFs, ask in Chinese or English

Choose a tag to compare

@Chuck0816 Chuck0816 released this 12 Nov 21:56
1b780d9

PolicyDesk now builds each company's knowledge base from the policy PDFs it uploads. Tables and figures are parsed, employees can ask in Chinese or English, and the web app has been redesigned around two views: Ask and Documents.

Highlights

  • Upload your own PDFs. PDFs are parsed with MinerU 4.x; figures are described by Qwen-VL (qwen3-vl-plus). New documents are added to the existing index, with no full rebuild.
  • Figures that fail are skipped, not fatal. The rest of the PDF is still added, and the app lists each skipped figure's page, caption, and reason.
  • Add, list, and delete documents from the web app or the CLI. Deletion removes a document from Chroma, MongoDB, and Redis together. The same file cannot be added twice.
  • Ask in Chinese or English. Query rewriting and answers follow the question's language, and keyword search now matches Chinese text.
  • Redesigned web app. Source cards show file, page, and type (text, table, figure) before the streamed answer. The Documents view shows upload progress and a status table with confirmed deletion. Chinese/English interface, light and dark themes, phone layout.

⚠️ Upgrade notes

  • Use a new knowledge base. The default KB_NAME is now policydesk_uploads_v1. Set a new, unused KB_NAME in .env; older namespaces still contain the sample policies.
  • Sample policies are no longer indexed. Files in data/ are kept only as test fixtures.
  • Install MinerU separately. Install MinerU 4.x in its own environment and set MINERU_COMMAND to the full path of its mineru executable, for example /opt/anaconda3/envs/mineru/bin/mineru.
  • CLI changes. preview, build, and --category were removed. Use add, delete, list, ask, and chat.
  • New optional settings: MINERU_PYTHON, MINERU_TIER, VL_MODEL.

Getting started

git pull
python -m pip install -r requirements.txt
docker compose up -d --wait
python -m streamlit run app.py

PolicyDesk v0.2.0 — Scoped Advanced RAG

Choose a tag to compare

@Chuck0816 Chuck0816 released this 08 Nov 05:25
a58f3ca

PolicyDesk v0.2.0 introduces an advanced retrieval pipeline with business metadata, optional policy scopes, hybrid search, and local relevance reranking.

Employees can search the full policy library or select a specific category, with inspectable policy excerpts supporting each answer.

Added

  • Business metadata: validated policy IDs, titles, versions, owners, document types, languages, and categories inherited by indexed nodes.
  • Scoped retrieval: category and document-type filters applied consistently to Chroma vector search and the BM25 corpus.
  • Query rewriting: one English retrieval query with numeric-preservation checks and fallback to the original question.
  • Hybrid search: semantic retrieval and BM25 combined through reciprocal rank fusion.
  • Local reranking: BGE selects up to four relevant excerpts before answer generation.
  • Scope controls: category selection in the Streamlit interface and CLI, with separate CLI conversation histories for each scope.
  • Source details: policy IDs, versions, categories, and reranker scores displayed alongside retrieved excerpts.

Fixed and improved

  • Aligned retrieval helpers and category-aware service interfaces.
  • Reused cached models across scope-specific web query engines.
  • Cleared previous web answers when changing search scope.
  • Rejected empty model output instead of displaying success or recording an empty chat answer.
  • Restricted Chroma collection creation to index-building operations.
  • Expanded regression tests and updated dependency, configuration, and setup documentation.

Upgrade notes

Install the updated dependencies from requirements.txt.

Existing indexes do not automatically receive business metadata. Choose a new KB_NAME, run python main.py build, and restart Streamlit. Preserve existing database and Docker volume settings to retain previous indexes and conversations.

Current limitations

PolicyDesk remains a simplified local application. Category filters control search scope and do not provide access control. Policy updates require a new index namespace, and version metadata does not automatically resolve superseded policies.

Qwen inference requires a valid DashScope API key and available account quota. Reranker scores indicate relative relevance, not answer confidence.

PolicyDesk v0.1.0 — Basic RAG Policy Assistant

Choose a tag to compare

@Chuck0816 Chuck0816 released this 03 Nov 17:37

Overview

The first release of PolicyDesk, a lightweight company policy assistant built with LlamaIndex.

Features

  • Policy question answering powered by Qwen Plus.
  • Local HuggingFace embeddings for document retrieval.
  • Persistent storage using MongoDB, Chroma, and Redis.
  • Streamlit interface with streamed answers and retrieved policy sources.
  • Named conversation sessions with saved history.

Getting Started

Follow the README to install dependencies, configure your DashScope API key, start the database services, and build the policy index.

Current Scope

This release focuses on a basic RAG workflow. Pre-retrieval processing, hybrid retrieval, and reranking are planned for future releases.

Answers should be checked against the retrieved policy sources.