💡 Actions Data Stream Adoption & CI Observability Standard

[github-actions[bot]]

Summary

Define an org-wide standard for adopting GitHub's new Actions Data Stream feature, which provides real-time execution telemetry streamed to external sinks (S3, Azure Event Hub) with at-least-once delivery. This decouples CI audit trails from mutable workflow run logs and unlocks downstream observability for agent costs, canary health, and security forensics.

Market Signal

GitHub's 2026 security roadmap introduces the Actions Data Stream as a core pillar — delivering workflow and job execution details across repositories and organizations to external systems. The feature provides an unalterable record of what ran and what it did, addressing a core incident response gap. Early access is available now, with GA expected within 6-9 months. The industry is converging on treating CI/CD as a production system requiring the same observability standards as application infrastructure.

User Signal

The org already has multiple observability-related ideas (cost dashboards #396/#338, agent activity audit logs #273) but lacks the telemetry infrastructure to implement them. The current approach of parsing run logs via gh api is fragile — issue #803 required hardening _run_json with exponential jittered backoff due to API failures. Multiple fleet monitoring issues (#894, #830) demonstrate the need for durable, queryable execution data that doesn't depend on mutable run logs.

Technical Opportunity

The canary rollout engine (scripts/lib/canary-rollout.sh) already collects run data via gh api calls. A Data Stream adoption standard would provide a standardized, durable sink for all execution telemetry — replacing fragile API-scraping with push-based streaming. This is foundational infrastructure that enables the downstream ideas already tracked: cost attribution (#396), agent audit log (#273), unified dashboard (#338). The streaming data format is structured JSON, directly queryable via Athena (S3) or KQL (Azure Data Explorer).

Assessment

Dimension	Score	Rationale
Feasibility	med	Depends on GitHub's GA timeline; early access available now. Requires an S3 bucket or Azure Event Hub — minimal infrastructure.
Impact	high	Foundational layer that unblocks multiple downstream observability ideas. Eliminates API rate-limit fragility in canary rollout and fleet monitoring.
Urgency	med	Early access available today. The org's observability backlog is growing (6+ related idea discussions). Position for GA adoption.

Adversarial Review

Strongest objection: We're a small org — S3/Azure Event Hub infrastructure is overkill. GitHub's built-in run logs and the gh API are sufficient for our scale.

Rebuttal: The current gh api approach is already proving fragile at current scale (issue #803 hardened _run_json with backoff). Data Stream provides durable, queryable data with zero API rate-limit cost. Even for a small org, the streaming cost is minimal (pennies/month for execution metadata) and it unlocks the entire observability layer the org has been ideating about for months. Start with S3 in audit-only mode and build dashboards incrementally.

Suggested Next Step

Apply for the Actions Data Stream early access program. Draft a standards/actions-data-stream.md defining the streaming target, retention policy, and schema expectations. Wire a proof-of-concept S3 sink and validate that canary rollout execution data appears in queryable form.