💡 Autonomous Agent Runaway Protection & Session Kill Switch

[github-actions[bot]]

Summary

With Copilot Desktop App GA (Jun 17), Claude Code Routines (cloud-hosted autonomous agents), and AI agents consuming GitHub Actions minutes (Copilot code review billing change Jun 1), the org faces a new risk category: runaway autonomous agent sessions that consume resources without human oversight. This standard defines per-session time/cost limits, a kill switch mechanism for terminating runaway agents across the fleet, and alerting when agents exceed expected resource envelopes.

Market Signal

• Uber exhausted its entire 2026 AI budget by April — a cautionary tale for organizations scaling AI agent usage
• Linux Foundation launched the Tokenomics Foundation to standardize AI billing metrics
• New governance tools emerging: Bifrost (inline token budget enforcement), Finout CostGuard (policy-based AI spend enforcement), Pay-i (pure-play token cost platform)
• Copilot code review now consumes Actions minutes (Jun 1 billing change) — adding a hidden cost multiplier to every PR
• Research finding: engineers using the most AI tokens achieve ~2x productivity but consume 10x tokens, questioning per-agent ROI
• Enterprise AI Controls GA (GitHub): audit logs now include actor_is_agent identifier for agent-initiated actions

User Signal

• The org already experienced cascade failure: dev-lead concurrency-group cancellation bug (dev-lead: issue-pickup runs cancelled by concurrency group → 111 labeled issues stuck with no PR #402) caused 111 labeled issues stuck with no PR — demonstrating that one workflow malfunction can create fleet-wide impact
• Multi-label issue creation race (fix(dev-lead): multi-label issue creation can cancel the dev-lead dispatch (race) #443) can cancel dev-lead dispatches
• The org runs 5+ autonomous agent workflows: dev-lead, compliance-audit, feature-ideation, initiative-driver, fleet-monitor
• Current controls are workflow-level (concurrency groups, job timeouts), not agent-session-level — a runaway session within its timeout window is invisible
• No per-agent resource accounting exists

Technical Opportunity

Several mechanisms are available today:

1. GitHub Actions workflow timeouts — timeout-minutes per job and step (already used, but not systematically audited)
2. OIDC-scoped ephemeral tokens with TTL — limit the blast radius of a compromised or runaway session
3. GitHub API audit logs — Enterprise AI Controls GA adds actor_is_agent to filter agent-initiated actions
4. Claude Code Routines — support session timeout configuration natively
5. Fleet-tracker workflow — already monitors workflow health; extend to agent session duration/cost tracking
6. Budget enforcement pattern — governance layer between agent code and LLM APIs that evaluates cost before each call and terminates at ceiling

Assessment

Dimension	Score	Rationale
Feasibility	med	Requires coordination across multiple agent platforms (Actions, Routines, Desktop App)
Impact	high	Prevents resource cascade failures and uncontrolled cost accumulation
Urgency	med	No immediate incident, but the fleet is growing and the Uber precedent is cautionary

Adversarial Review

Strongest objection: The org is small (internal DevX, 3–4 downstream repos). Runaway risk is manageable with existing workflow concurrency controls and job-level timeouts. Over-engineering protection adds complexity without proportionate value.

Rebuttal: The org is scaling agent automation rapidly — from 2 agent workflows 3 months ago to 5+ today. The concurrency-group bug (#402) already demonstrated cascade failure at current scale. One runaway agent dispatching across repos creates exponential work: each compliance finding triggers a dev-lead run, which may create PRs that trigger CI workflows. Proactive per-session limits are cheaper than reactive cleanup. The fleet-tracker is already in place — extending it to duration/cost alerts is incremental, not over-engineering.

Suggested Next Step

1. Document the org's agent fleet inventory with expected resource envelopes (max duration, max API calls, max Actions minutes per session per workflow)
2. Audit all agent workflow timeout-minutes settings for consistency
3. Implement fleet-tracker alerts for sessions exceeding 2x expected duration
4. Evaluate Bifrost or similar inline budget enforcement for Claude Code token spend