[observability] Observability Coverage Report - 2026-09-20

[github-actions[bot]]

Executive Summary

This report analyzes a representative sample of 20 workflow runs from the last 7 days (sample fetched from the most recent 30 runs, then capped per policy). Overall observability coverage is strong for both AWF Firewall and MCP Gateway telemetry in this sample, with no run missing critical telemetry for enabled components.

For firewall-enabled runs, all sampled workflows that emitted firewall activity also showed recorded allow/deny evidence in aggregated firewall telemetry, including blocked events where expected. For MCP-enabled runs, tool execution telemetry was consistently available in the sampled run set via MCP usage data, and no sampled MCP-enabled run was identified with complete telemetry absence.

One operational limitation affected deep per-run artifact validation in this environment: downloaded run folders under /tmp/gh-aw/aw-mcp/logs/ were root-owned and not readable by this agent process, so quality checks are based on the MCP logs tool’s structured aggregate outputs rather than direct file-by-file content parsing.

Key Alerts and Anomalies

No critical observability gaps were detected in the analyzed sample.

🔴 Critical Issues:

• No sampled firewall-enabled run was found without firewall telemetry evidence.
• No sampled MCP-enabled run was found missing both gateway and RPC telemetry evidence.

⚠️ Warnings:

• Local filesystem access to /tmp/gh-aw/aw-mcp/logs/run-* was denied due to ownership/permissions, preventing direct per-run path verification for access.log, gateway.jsonl, and rpc-messages.jsonl in this execution context.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log-equivalent evidence)	20	20	100.0%	✅
MCP Gateway (gateway.jsonl or rpc-messages.jsonl-equivalent evidence)	20	20	100.0%	✅

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Daily Documentation Healer	35545349481	✅ (aggregated)	55	54	1	✅
Daily Go Test Parallelizer	35543787813	✅ (aggregated)	56	52	4	✅
Contribution Check	35542916499	✅ (aggregated)	109	109	0	✅
Issue Monster	35545308318	✅ (aggregated)	18	18	0	✅
PR Sous Chef	35544680472	✅ (aggregated)	32	32	0	✅

Missing Firewall Logs (access.log)

No missing firewall telemetry detected in analyzed runs.

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Daily Documentation Healer	35545349481	MCP usage aggregate	present	github,safeoutputs	present	unknown	✅
Issue Monster	35545308318	MCP usage aggregate	present	github,safeoutputs	present	unknown	✅
PR Sous Chef	35544680472	MCP usage aggregate	present	github,safeoutputs	present	unknown	✅
Agent Job Health Monitor	35544205208	MCP usage aggregate	present	github,safeoutputs	present	unknown	✅
Contribution Check	35542916499	MCP usage aggregate	present	github,safeoutputs	present	unknown	✅

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

No sampled MCP-enabled run was identified with complete MCP telemetry absence.

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total firewall requests analyzed: 352
• Domains accessed: 18 unique domain:port targets
• Blocked requests: 5 (1.4%)
• Most accessed domains: api.githubcopilot.com:443, api.anthropic.com:443, api.openai.com:443

Gateway Log Quality

• Telemetry source: MCP usage aggregate (server/tool call metadata)
• Total MCP tool calls analyzed: 163
• MCP servers used: github, mcpscripts, safeoutputs
• Total tool calls by server: github=100, mcpscripts=3, safeoutputs=60
• Error-rate precision is limited in aggregate view; no sample-level hard telemetry absence detected

Healthy Runs Summary

The sampled set indicates broad observability health across scheduled workflows, including both success and failure outcomes. The highest anomaly signal in this sample is operational drift/failure concentration (not telemetry absence), notably in agent-job-health reliability.

Recommended Actions

1. Fix permissions on /tmp/gh-aw/aw-mcp/logs/run-* artifacts so agents can directly validate access.log, gateway.jsonl, and rpc-messages.jsonl path presence and schema quality per run.
2. Keep current firewall telemetry behavior; blocked-request visibility is working and useful (e.g., ab.chatgpt.com:443 denials surfaced correctly).
3. Add a lightweight post-run telemetry manifest (per run) that explicitly records discovered firewall and MCP log file paths to remove ambiguity across layout variants.

📊 Historical Trends

This execution used a capped representative sample from recent runs and did not perform a longitudinal multi-day trend join beyond the downloaded window slice.

---

Report generated automatically by the Daily Observability Report workflow
Analysis window: Last 7 days | Runs analyzed: 20

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · codex · gpt53codex · 14.2 AIC · ⌖ 5.03 AIC · ⊞ 15.4K · ◷

• expires on Sep 21, 2026, 3:48 PM UTC-08:00