Repository navigation
Highlights
PolicyDesk now has a second domain for SRE incident troubleshooting, next to the existing policy assistant. The policy domain is unchanged and remains the default.
With KB_DOMAIN=oncall, PolicyDesk combines a service catalog in Neo4j, runbooks and postmortems in Markdown, and GraphRAG retrieval to answer questions like:
- If product-catalog is down, which services are affected?
- checkout latency is up and payment errors are rising. What is the likely root cause?
- Who owns valkey-cart and where do we escalate?
- Has this Kafka consumer lag happened before? How was it fixed?
What's new
Knowledge graph
- Service catalog import:
service_catalog.yamlandalerts.yamlare validated and loaded into Neo4j without an LLM, covering services, datastores, teams, dependencies, owners, escalation paths, and alerts linked to runbook sections. - Postmortem extraction: Qwen extracts incidents, affected services, root causes, symptoms, and mitigations with a strict schema. Extracted names are aligned to the catalog, including Chinese aliases, and symptoms to a controlled vocabulary. Results are cached in Redis.
- Safe provenance: every graph fact records the documents and chunks that support it. Deleting a postmortem removes only facts that no other document supports. A failed extraction changes nothing, and a failed graph write rolls back.
Retrieval
- Three modes:
hybrid(the existing chain, used as the baseline),graph, andhybrid+graph. - Deterministic entity linking on the original question and fixed Cypher templates; the LLM never writes Cypher.
- Root-cause ranking based on shared downstream dependencies, distance, and incident history.
- Graph facts keep reserved slots after reranking, so the reranker cannot drop them.
- The catalog is also rendered as text, so the hybrid baseline sees the same facts and the comparison is fair.
Interface
- Web demo: mode switch, side-by-side comparison (hybrid vs hybrid + graph), separate document evidence and graph reasoning paths, catalog import, Markdown upload, and alert updates without re-uploading the catalog.
- CLI:
import-catalog, Markdownadd, and--mode. - Metrics: retrieval, first-token, generation, and total time, plus LLM calls and estimated tokens, for every answer.
- Structured output: each answer ends with a JSON block (
root_cause_services,affected_services,owner_team,uncertain) for evaluation.
Demo corpus
examples/oncall_demo/ contains a 16-component shop catalog, 5 alerts, 5 runbooks, 5 postmortems, and 16 test questions with expected answers.
Upgrade
pip install -r requirements.txt
docker compose --profile oncall up -d --wait