feat: Log Streams — stream audit logs and secret events to Datadog

[nimish-ks]

🔍 Overview

This PR adds Log Streams: continuous, near-real-time export of organisation audit logs and app secret events to external log platforms, starting with Datadog. Streams are configured under Integrations → Log Streams, ship in the background, and expose per-delivery observability with manual retry/backfill for anything that fails.

💡 Proposed Changes

• New LogStream + LogStreamDeliveryEvent models: provider config, event-source selection, per-source cursors, health, and a per-chunk delivery history.
• Shipping engine on a dedicated log-streams RQ queue (30s scheduler sweep). Events are serialized into a versioned, OTel-aligned envelope, chunked under the destination's payload limits, and delivered with an exponential retry ladder honouring Retry-After.
• At-least-once delivery: cursors advance only after the destination accepts a chunk; dedupe on the stable @event.id. Failed chunks are recorded with their exact event range and never block newer events; auth failures pause the stream; ranges older than Datadog's 18h ingestion window are recorded as skipped, with a REST backfill path.
• Secret keys, values and comments are not part of exported logs. Events reference secrets by ID and path — a new ⌘K lookup resolves a pasted secret ID straight to its app/environment/path.
• Event sources and providers are registries (org_audit, secrets today), so new sources and providers slot in without schema changes.
• Datadog provider: logs intake with gzip, standard-attribute remapping (network.client.ip, http.useragent, usr.*, ddtags) so facets and Cloud SIEM rules work with no pipeline config; Datadog site regions as a dropdown in the credential form; Test connection validates the API key without writing any data.
• New LogStreams permission resource; managing streams requires a role with global access. Log-stream changes appear in the org audit log with their own filter tab.
• Routes the audit logs REST API at /v1/logs/audit/ for backfills and custom tooling.
• New log stream worker pool, sized via LOG_STREAM_WORKERS (default: 2).
• Zero new dependencies.

📦 Exported event shape

{
  "schema_version": 1,
  "event": { "id": "…", "category": "secrets", "type": "read" },
  "timestamp": "2026-07-30T12:00:00+00:00",
  "actor": { "type": "service_account", "id": "…", "name": "ci-deploy", "token": { "name": "gh-actions" } },
  "usr": { "id": "…", "name": "Dev Eloper", "email": "dev@example.com" },
  "network": { "client": { "ip": "203.0.113.7" } },
  "http": { "useragent": "phase-cli/1.18" },
  "phase": {
    "organisation": { "id": "…", "name": "acme" },
    "app": { "id": "…", "name": "backend" },
    "environment": { "id": "…", "name": "Production", "type": "PROD" },
    "secret": { "id": "…", "path": "/api/payments", "version": 3, "type": "secret" },
    "description": "Secret read in backend / Production by ci-deploy"
  },
  "ddsource": "phase",
  "service": "phase-console",
  "ddtags": "phase_org:acme,phase_stream:datadog-prod"
}

Organisation audit events ("category": "org_audit") carry a phase.resource block (type, id, metadata) plus old_values / new_values instead of the secret block.

🖼️ Screenshots or Demo

💚 Did You...

• Ensure linting passes (code style checks)?
• Update dependencies and lockfiles (if required) (N/A — zero new dependencies)
• Update migrations (0132 log streams, 0133 delivery-range id bounds)
• Regenerate graphql schema and types
• Verify the app builds locally?
• Manually test the changes on different browsers/devices?