Skip to content

release-2026.06-rag-preview

Pre-release
Pre-release

Choose a tag to compare

View all tags
@dongkoony dongkoony released this 01 Jun 04:50
· 6 commits to main since this release
release-2026.06-rag-preview
513c710
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Channel

Preview

Summary

This release introduces the first RAG preview layer for the DevOps Incident Triage Model.

The project remains classifier-first. The existing Transformer-based incident classifier is kept intact, while this release adds a preview retrieval layer that can retrieve evidence from domain-specific runbook documents.

LLM-assisted remediation generation is not included yet. That will be introduced in a later incident-assist beta release.

Included

  • POST /retrieve FastAPI endpoint
  • Domain-aware runbook retrieval flow
  • Local TF-IDF sparse retrieval index for preview use
  • Evidence-based retrieval response schema
  • Section-level citations from runbook documents
  • Retrieval metadata including index type, embedding model placeholder, and latency
  • Prometheus-style retrieval metrics:
    • ditri_retrieval_requests_total
    • ditri_retrieval_latency_seconds
  • RAG roadmap documentation
  • RAG evaluation plan
  • Runbook placeholder document structure
  • Release evidence document for release-2026.06-rag-preview

Validation

GitHub Actions on main: Success

Local validation:

uv run --extra dev --extra api ruff check .
uv run --extra dev --extra api pytest -q

Result:

ruff check: passed
pytest: 47 passed, 10 skipped

FastAPI smoke validation:

GET /health     -> 200
POST /retrieve  -> 200
GET /metrics    -> 200

Known Limitations

  • This is a preview release, not a production RAG backend.
  • Retrieval currently uses a local TF-IDF sparse index, not a production Vector DB.
  • /assist and LLM-generated remediation guidance are not implemented yet.
  • Runbooks are portfolio-grade placeholders, not real production incident records.
  • The current dataset remains synthetic.
  • Retrieval quality still needs formal evaluation with retrieval hit rate, groundedness, citation coverage, and hallucination checks.

Next Release Direction

The next planned release is:

release-2026.07-incident-assist-beta

Planned focus:

  • Classifier + RAG integration
  • POST /assist API design and implementation
  • LLM-generated remediation guidance
  • Root cause candidate generation
  • Evidence citations from retrieved runbooks
Assets 2
Loading