Skip to content

v3.8.0

Latest

Choose a tag to compare

@placerda placerda released this 20 Jul 18:47
e43f24e

Operators can now opt into versioned audit evidence for orchestrator requests,
routes, grounding sources, tools, and outcomes through the existing OpenTelemetry
and Application Insights pipeline. Existing deployments keep their current
behavior: auditing is disabled by default, and enabled auditing is metadata-only
by default.

This release implements
Azure/GPT-RAG#571 through
PR #277.

Added

  • Privacy-conscious audit evidence. Fixed-name gptrag.audit.* events use
    bounded metadata, keyed pseudonyms, recursive prohibited-data filtering,
    explicit cancellation and partial-output states, and omission counters.
    Audit-only export does not enable normal application log export.
  • Request correlation. Successful orchestrator and feedback responses
    include a server-generated X-Correlation-ID, with additive Cosmos turn
    correlation metadata. Inbound correlation IDs are ignored.
  • Shared v1 contract. Reusable logical and Application Insights wire schemas
    define the orchestrator events and reserve the canonical ingestion taxonomy:
    ingestion.run.started, ingestion.run.completed,
    ingestion.run.failed, ingestion.run.cancelled,
    ingestion.document.indexed, ingestion.document.rejected, and
    ingestion.document.deleted. These ingestion names are reserved for component
    integration and are not emitted by this orchestrator release.

Privacy and configuration

  • Metadata-only default. AUDIT_EVENTS_ENABLED=false remains the default.
    When enabled, the operator must provide a 256-bit AUDIT_HMAC_KEY; raw
    identifiers are not exported.
  • Sensitive content requires explicit opt-in. Prompts, responses, source
    excerpts, tool arguments, and tool results are considered only when
    AUDIT_SENSITIVE_CONTENT_ENABLED=true and
    AUDIT_SENSITIVE_CONTENT_FIELDS contains the approved fields. The empty
    allowlist captures none. Secret filtering remains defense in depth, not a
    data-loss-prevention boundary.
  • Rollback. Set AUDIT_EVENTS_ENABLED=false and restart the orchestrator to
    stop new audit events. Previously exported telemetry is not deleted.

Evidence limitations

  • These events are best-effort operational evidence. They are not an immutable
    ledger and do not establish legal or regulatory compliance.
  • Fixed budgets can omit detail events while request terminal events report
    omission counts. Azure Monitor asynchronous delivery has no application
    callback, so background delivery failures cannot be confirmed by the service.
  • Dependency or request-body validation failures before endpoint handling may
    not include X-Correlation-ID.
  • Foundry IQ MCP start times reconstructed from completion activity are
    approximate and do not prove when the remote tool began execution.
  • Application Insights stringifies custom properties and drops logical nulls.
    Consumers must apply the documented wire conversion, including decoding the
    reserved all-zero parent sentinel to logical null before validation.

Validation

Validation was completed against the exact reviewed commit
844fe14757d07a2cdc828189105fbce831f3c11d:

  • Full test suite: 533 passed.
  • Linux AMD64 Docker build: passed.
  • Ruff on every Python file changed by PR #277: passed.
  • python -m compileall -q src tests scripts: passed.
  • pip check in the freshly built pinned image: passed.
  • Committed logical and Application Insights schema raw-byte SHA-256 hashes:
    independently reproduced.

This release does not publish a container image and did not perform an Azure
deployment or resource deletion.