[daily regulatory] Regulatory Report - 2026-05-24

[github-actions[bot]]

Eight daily report discussions published on 2026-05-24 were reviewed and cross-checked for data consistency, quality, and anomalies. Overall data quality is good — no critical numeric discrepancies were found between metrics of identical scope across reports. Three significant issues require attention: (1) a P0 incident (Codex CLI 0.133.0 streaming regression, #34522) is active; (2) the Copilot session completion rate collapsed to 2% (from 44% the prior day), with 98% of sessions ending in action_required within seconds; (3) the OAuth token for conversation-log fetching is missing, degrading the Copilot Session Insights analysis to metadata-only.

The repository is in an unusual operational mode: of 48 PRs in the 24-hour window, 47 were authored by bots (28 Copilot SWE, 20 github-actions, 1 Codex self-review) with a single human commit (Go 1.26 upgrade). The agentic self-maintenance loop is functioning, but the Codex CLI incident and session-completion regression are the two highest-priority open items.

📋 Full Regulatory Report

📊 Reports Reviewed

#	Report	Discussion	Created	Category	Status
1	Daily Performance Summary	#34535	21:12 UTC	Audits	✅ Valid
2	Daily Team Evolution Insights	#34528	20:35 UTC	Audits	✅ Valid
3	Daily Code Metrics	#34519	18:58 UTC	Audits	✅ Valid
4	Copilot Agent Analysis	#34514	18:39 UTC	Audits	✅ Valid
5	Daily Secrets Analysis	#34499	17:59 UTC	Audits	✅ Valid
6	Security Observability	#34481	16:31 UTC	⚠️ General	⚠️ Wrong category
7	GitHub API Consumption	#34414	11:35 UTC	Audits	✅ Valid
8	Copilot Session Insights	#34397	08:01 UTC	Audits	⚠️ Degraded
9	Safe Output Health	#34375	05:41 UTC	Audits	✅ Valid

i️ Note: #34482 (GEO Audit), #34398 (Experiments), and others were identified but not deeply analyzed to stay within scope.

---

🔍 Data Consistency Analysis

Cross-Report Metrics Comparison

Metric	Performance	Team Evolution	Agent Analysis	Scope Match	Status
PRs merged (24h)	53 (of 100 sampled)	28 (true 24h)	32 agent PRs (72.7%)	⚠️ Different scopes	i️ See Note 1
Open PRs	8 (of 100 sampled)	4 (true 24h end)	—	⚠️ Different scopes	i️ See Note 1
Open Issues	51 (of 500 sampled)	13 opened today	—	⚠️ Different scopes	i️ See Note 2
Workflow runs	211 runs (API Report, 24h)	—	—	Single source	✅
Safe output writes	26 (API Report, 24h)	—	8 messages (prior day)	⚠️ Different windows	i️ See Note 3
Agent sessions	50 (Session Insights)	—	44 agent PRs	⚠️ Different metrics	✅ Compatible

Scope Notes:

• Note 1: Performance Summary samples 100 most recent PRs (historical snapshot); Team Evolution tracks the true last-24h window (48 PRs total, 28 merged). Not a discrepancy — different sampling strategies.
• Note 2: Performance Summary samples 500 most recent issues; Team Evolution counts issues opened/closed today. Same expected behavior.
• Note 3: API Consumption reports 26 safe-output writes for 2026-05-24; Safe Output Health covers the 2026-05-23 window (8 messages across 6 runs). Sequential days, no overlap — consistent.

Internal Consistency Checks

Report	Check	Result
Performance Summary	Open (8) + Closed (39) + Merged (53) = 100 PRs	✅ Passes
Performance Summary	Open (51) + Closed (449) = 500 Issues	✅ Passes
API Consumption	Success (167) + Failed (44) = 211 runs	✅ Passes (79.1% ≈ 167/211)
Agent Analysis	Merged (32) / Total (44) = 72.7%	✅ Passes
Security Observability	Allowed (127) + Blocked (27) = 154 requests	✅ Passes
Safe Output Health	Success (8) / Total (8) = 100%	✅ Passes

Consistency Score

• Overall Consistency: 95% (all in-scope same-metric comparisons pass)
• Critical Discrepancies: 0
• Minor Discrepancies / Scope Mismatches: 3 (all expected by design)
• Category Placement Issue: 1 (Security Observability in General instead of Audits)

---

⚠️ Issues and Anomalies

🔴 Critical Issues

1. P0 Active Incident — Codex CLI 0.133.0 Streaming Regression
   • Affected Reports: Team Evolution Insights ([daily-team-evolution] 🌱 Daily Team Evolution Insights — 2026-05-24 #34528)
   • Description: Codex CLI 0.133.0 introduced a stream_options.include_obfuscation regression that broke streaming. Filed as P0 at 19:10 UTC with pin-back issues [aw-failures] 6h failure cluster (2026-05-24 19:10): Codex CLI 0.133.0 stream_options.include_usage regression broke 7 workflo [Content truncated due to length] #34521 and [aw-failures] [fix][P0] Pin Codex CLI back from 0.133.0 — stream_options.include_usage rejected by gpt-5.5 #34522.
   • Severity: Critical (active incident)
   • Recommended Action: Confirm pin-back has been applied. Track [aw-failures] [fix][P0] Pin Codex CLI back from 0.133.0 — stream_options.include_usage rejected by gpt-5.5 #34522 to closure.

🟠 High-Priority Issues

2. Copilot Session Completion Rate Collapse (2% from 44%)

   • Affected Reports: Copilot Session Insights ([copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-05-24 #34397)
   • Metric: session_completion_rate
   • Expected: ~44% (2026-05-23 recovery day)
   • Actual: 2.0% — 49 of 50 sessions ended in action_required within 0 seconds
   • Severity: High
   • Root Cause Hypothesis: Activation/permission gating change between 2026-05-23 and 2026-05-24 (workflow approval policy, secret rotation, or branch protection)
   • Recommended Action: Investigate gate state diff between the two days. Check workflow approval policy changes.

3. OAuth Token Missing — Session Log Fetching Degraded

   • Affected Reports: Copilot Session Insights ([copilot-session-insights] Daily Copilot Agent Session Analysis — 2026-05-24 #34397)
   • Description: copilot-session-data-fetch step failed: this command requires an OAuth token. Re-authenticate with: gh auth login. 0/50 conversation transcripts downloaded.
   • Severity: High (reduces behavioral analysis to metadata-only)
   • Recommended Action: Restore the GitHub OAuth token in the workflow secrets. This is the second contributing factor to today's analysis degradation.

🟡 Warnings

4. Firewall Coverage Critically Low (1 workflow in 7 days)

   • Affected Reports: Security Observability ([security-observability] Daily Security Observability Report — 2026-05-24 #34481)
   • Metric: firewall_enabled_workflows = 1
   • Description: Only one firewall-enabled workflow run was observed in the 7-day analysis window. 17.5% block rate with all blocked requests having unresolved destinations.
   • Recommended Action: Enable firewall on more workflows. Investigate the 27 (unknown) blocked requests.

5. Test-to-Source Ratio Declining

   • Affected Reports: Daily Code Metrics ([daily-code-metrics] Daily Code Metrics Report - 2026-05-24 #34519)
   • Metric: test_to_source_ratio = 2.057 (−1.9% in 7d, −5.6% in 30d)
   • Description: Source LOC (+4.9%/7d) growing faster than test LOC (+2.8%/7d). Large files grew 74% in 7 days.
   • Recommended Action: Monitor test ratio trend. Allocate explicit test-writing work for new pkg/workflow and pkg/parser features.

6. Security Observability Discussion in Wrong Category

   • Affected Reports: Security Observability ([security-observability] Daily Security Observability Report — 2026-05-24 #34481)
   • Description: Filed in General category instead of Audits where all other daily reports reside.
   • Severity: Low (administrative)
   • Recommended Action: Fix workflow to use the Audits category.

7. API Consumption Well Below 7-Day Average

   • Affected Reports: API Consumption ([api-consumption] 📊 GitHub API Consumption Report — 2026-05-24 #34414)
   • Metric: 22,295 calls vs. 7-day average ~44,184
   • Description: Today's API consumption is ~50% of the rolling average. Likely correlated with the session completion rate collapse (fewer sessions = fewer API calls). Not an anomaly in isolation, but corroborates the gate-regression hypothesis.
   • Severity: Informational

---

📈 Trend Analysis

Metric	2026-05-20	2026-05-21	2026-05-22	2026-05-23	2026-05-24	Trend
Session completion rate	0%	12%	2%	44%	2%	⚠️ Volatile
Safe output success rate	—	97.9%	95.8%	100%	100%	✅ Stable
Agent PR merge rate	—	—	—	82%	72.7%	⚠️ Declining
API calls/day	—	—	—	~44K avg	22K	↓ Below avg

Notable Trends:

• Session completion rate is highly volatile — 5 data points show 0%, 12%, 2%, 44%, 2%. The single good day (2026-05-23) appears to be an outlier, not a trend.
• Safe output health is consistently excellent (100% for 2 days, near-100% prior).
• Agent PR merge rate is declining over 3 days (88% → 82% → 72.7%) — warrants watching.

---

📝 Per-Report Analysis

View all per-report details

Daily Performance Summary (#34535)

Time Period: Sampled 100 PRs / 500 issues / 100 discussions (most recent)
Quality: ✅ Valid

Metric	Value	Validation
Total PRs sampled	100	✅
Merged PRs	53 (53%)	✅ Math checks out
Open PRs	8 (8%)	✅
Avg PR merge time	1.8 hours	✅
Total issues sampled	500	✅
Closed issues	449 (89.8%)	✅
Open issues	51 (10.2%)	✅
Discussion answer rate	0%	⚠️ Low engagement

Daily Team Evolution Insights (#34528)

Time Period: Last 24 hours (true window)
Quality: ✅ Valid

Metric	Value	Validation
Commits on main	41	✅
PRs in window	48	✅
PRs merged	28 (58.3%)	✅
PRs closed without merge	16	✅ (48−28−4 open = 16)
Active contributors	4	✅ (Copilot, github-actions, Codex, pelikhan)
P0 incidents	1 (Codex CLI #34522)	🔴 Active
Integrity filter blocks	6 items	⚠️ Note

Note: 6 items blocked by DIFC integrity filter in this run — agent requires "approved" integrity. Issues #32481 and #33641 were blocked. This is functioning-as-designed but indicates the filter is actively gating data.

Daily Code Metrics (#34519)

Time Period: 7-day and 30-day trends; historical window 33 days
Quality: ✅ Valid

Metric	Value	Validation
Total LOC	1,555,355	✅
7-day growth	+4.0%	✅
Quality score	62/100	✅
Test-to-source ratio	2.057	⚠️ Declining
Large files (>500 LOC)	572 (+74%/7d)	⚠️ Growing rapidly
7-day commits	441	✅

Copilot Agent Analysis (#34514)

Time Period: Last 24 hours
Quality: ✅ Valid

Metric	Value	Validation
Agent PRs total	44	✅
Merged	32 (72.7%)	✅
Closed without merge	12	✅ (44−32=12)
Avg duration	~93 min	✅
Codex auth churn PRs	3 closed	⚠️ Wasteful

Daily Secrets Analysis (#34499)

Time Period: Point-in-time snapshot
Quality: ✅ Valid

Metric	Value	Validation
Workflows analyzed	235	✅
Redaction coverage	100%	✅
Permission blocks	100%	✅
Secrets in job outputs	3 workflows	⚠️ Low risk per report
Unique secret types	36	✅

Security Observability (#34481)

Time Period: Last 7 days
Quality: ⚠️ Low coverage

Metric	Value	Validation
Firewall-enabled workflows	1	⚠️ Very low
Total requests	154	✅
Blocked requests	27 (17.5%)	⚠️ Unknown destinations
DIFC events	0	✅ (expected given low activity)

GitHub API Consumption (#34414)

Time Period: Last 24 hours
Quality: ✅ Valid

Metric	Value	Validation
Total runs	211	✅
Successful runs	167 (79.1%)	✅
REST API calls	22,295	✅
Safe-output writes	26	✅
Avg duration	216.8s	✅

Copilot Session Insights (#34397)

Time Period: 2026-05-24T06:51Z → 07:38Z (47-min slice)
Quality: ⚠️ Degraded (metadata-only — OAuth failure)

Metric	Value	Validation
Sessions analyzed	50	✅
Completion rate	2% (1/50)	🔴 Regression
action_required rate	98% (49/50)	🔴
Avg duration	0.15 min	⚠️ Near-zero
Conversation logs	0/50	🔴 OAuth failure

Safe Output Health (#34375)

Time Period: 2026-05-23 ~04:18 → 2026-05-24 ~05:30 UTC
Quality: ✅ Valid

Metric	Value	Validation
Runs analyzed	13	✅
SO jobs executed	6	✅
Messages processed	8	✅
Success rate	100%	✅
Active error clusters	0	✅

---

💡 Recommendations

Process Improvements

1. Standardize discussion categories: Security Observability should file in Audits, not General. All 8 other reports use Audits — one exception creates confusion for filtering/monitoring.

2. Cross-report metric scope documentation: The Performance Summary and Team Evolution reports both report PR/issue counts but with different sampling strategies. Adding a brief scope: annotation to each report's header would eliminate ambiguity without requiring redesign.

Data Quality Actions

1. Restore OAuth token for session log fetching (High priority): Without it, Copilot Session Insights is metadata-only. This has been flagged in the report itself — action is needed before the next run.

2. Investigate session completion regression (High priority): The 44% → 2% drop is the largest single-day regression in the 5-day history. Requires diff of activation policy/secrets between 2026-05-23 and 2026-05-24.

3. Resolve Codex auth churn (Medium priority): 3 PRs closed unmerged on Codex auth wiring (Fix Codex engine auth wiring for gateway session keys and add harness diagnostics #34470, Codex engine: standardize auth on OPENAI_API_KEY and remove CODEX_API_KEY path #34439, Codex: use CODEX_API_KEY for firewall-injected openai-proxy provider config #34425). Agent Analysis recommends a single consolidated spec before further attempts — implement this.

Workflow Suggestions

1. Expand firewall coverage: Only 1 of the ~235 compiled workflows had firewall enabled in the last 7 days. The Security Observability report itself calls this out. Set a target of ≥10 firewall-enabled runs/day.

2. Add circuit-breaker for repeated action_required on same branch: 20 runs hit copilot/update-cli-versions-one-more-time in 47 minutes with a 5% success rate. Pure CI burn.

---

📊 Regulatory Metrics

Metric	Value
Reports Reviewed	9
Reports Passed (✅)	6
Reports with Issues (⚠️)	3
Reports Failed (❌)	0
Critical Issues	1 (P0 active incident)
High-Priority Issues	2
Warnings	4
Internal Math Checks Passed	6/6
Cross-Report Discrepancies (same-scope)	0
Overall Health Score	78%

---

Report generated automatically by the Daily Regulatory workflow
Data sources: Daily report discussions from github/gh-aw (2026-05-24)
Run: §26373301900

References:

• §26373301900 — This regulatory run
• §26372662280 — Daily Performance Summary
• §26355346065 — Copilot Session Insights

Generated by ⚖️ Daily Regulatory Report Generator · sonnet46 2.2M · ◷

• expires on May 27, 2026, 9:38 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Daily Regulatory Report Generator.

A newer discussion is available at Discussion #34779.