Skip to content

v2.5.0

Latest
Latest

Choose a tag to compare

View all tags
@placerda placerda released this 20 Jul 20:20
v2.5.0
d95bc86
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Operators now receive a consistent metadata-only audit trail for every ingestion run and confirmed document outcome, while richer provenance remains opt-in and governance can fail closed when strict mode is enabled. This release completes the ingestion work for Azure/GPT-RAG#571 through PR #261.

User and operator impact

  • Base events are always emitted. The service emits ingestion.run.started/completed/failed/cancelled and ingestion.document.indexed/rejected/deleted through the existing OpenTelemetry/Application Insights pipeline even when provenance is disabled. Every run has exactly one started and one terminal event, cancellation is preserved, and document events come only from confirmed Azure AI Search results. Audit export remains bounded and best-effort, so telemetry failures do not turn successful ingestion work into failures.
  • Provenance remains opt-in. INGESTION_PROVENANCE_ENABLED defaults to false; metadata-only base events still emit, but richer provenance fields remain absent until this flag is enabled. INGESTION_REQUIRE_GOVERNANCE_METADATA defaults to false. INGESTION_DEFAULT_CLASSIFICATION defaults to unclassified, and INGESTION_DEFAULT_RIGHT_TO_USE defaults to not_asserted for non-strict provenance when a document omits those values.
  • Strict governance fails closed. With provenance enabled, INGESTION_REQUIRE_GOVERNANCE_METADATA=true requires documents to explicitly provide both classification and right-to-use. Configured defaults are never substituted as governance evidence. Enabling strict governance while provenance is disabled fails startup with an actionable configuration error.

Compatibility

  • Shared orchestrator v3.8.0 contract. Ingestion vendors the same audit-event-v1 logical and Application Insights wire schemas used by gpt-rag-orchestrator v3.8.0. The pinned SHA-256 hashes are:
    • Logical schema: 825db8ef40a81e2c19e5d80d37c565b6b47fc9a6540e9881d35cc12b8fde5aab
    • Application Insights wire schema: 066c8f5408610ab839d5121d06ca5bc59e8797e551d5c47c875c5ba52f7e0588
  • Exported records use queryable customEvents.name values under gptrag.audit.ingestion.*, and logical root events retain the required parent property through the contract root sentinel.

Privacy and retention limitations

  • source_uri_id and source_version_id are deterministic, unsalted SHA-256 references rather than raw paths, URLs, or filenames. They may be dictionary-matched for small candidate sets and are not secrets, proof of source or custody, or evidence of regulatory compliance. Checksums and audit metadata can also reveal document relationships, so protect Application Insights with access controls appropriate for the ingested content.
  • delete_after expresses downstream retention-policy intent only. It does not schedule or trigger automatic purge. Only a confirmed Azure AI Search deletion emits ingestion.document.deleted; downstream retention enforcement remains an operator responsibility.

Validation

  • The exact reviewed PR head, 5cfbd180bdf2fd1dcd816dbee483894a43c22cac, was merged after all reported CI checks completed successfully.
  • Independent final review found no blockers and validated actual Azure Monitor exporter envelopes against the pinned Application Insights wire schema for all seven event types.
  • All 47 focused audit contract, sanitizer, and emitter tests passed, including logical/wire hash pinning, strict-governance failure paths, sanitization, correlation, cancellation, and exported event names and parent properties.
Assets 2
Loading