Repository navigation
PolicyDesk v0.2.0 — Scoped Advanced RAG
PolicyDesk v0.2.0 introduces an advanced retrieval pipeline with business metadata, optional policy scopes, hybrid search, and local relevance reranking.
Employees can search the full policy library or select a specific category, with inspectable policy excerpts supporting each answer.
Added
- Business metadata: validated policy IDs, titles, versions, owners, document types, languages, and categories inherited by indexed nodes.
- Scoped retrieval: category and document-type filters applied consistently to Chroma vector search and the BM25 corpus.
- Query rewriting: one English retrieval query with numeric-preservation checks and fallback to the original question.
- Hybrid search: semantic retrieval and BM25 combined through reciprocal rank fusion.
- Local reranking: BGE selects up to four relevant excerpts before answer generation.
- Scope controls: category selection in the Streamlit interface and CLI, with separate CLI conversation histories for each scope.
- Source details: policy IDs, versions, categories, and reranker scores displayed alongside retrieved excerpts.
Fixed and improved
- Aligned retrieval helpers and category-aware service interfaces.
- Reused cached models across scope-specific web query engines.
- Cleared previous web answers when changing search scope.
- Rejected empty model output instead of displaying success or recording an empty chat answer.
- Restricted Chroma collection creation to index-building operations.
- Expanded regression tests and updated dependency, configuration, and setup documentation.
Upgrade notes
Install the updated dependencies from requirements.txt.
Existing indexes do not automatically receive business metadata. Choose a new KB_NAME, run python main.py build, and restart Streamlit. Preserve existing database and Docker volume settings to retain previous indexes and conversations.
Current limitations
PolicyDesk remains a simplified local application. Category filters control search scope and do not provide access control. Policy updates require a new index namespace, and version metadata does not automatically resolve superseded policies.
Qwen inference requires a valid DashScope API key and available account quota. Reranker scores indicate relative relevance, not answer confidence.