[observability] Observability Coverage Report - 2026-08-17

[github-actions[bot]]

Executive Summary

I analyzed a capped sample of 20 completed workflow runs from the last 7 days using the repository-wide agenticworkflows logs pull and the artifacts saved under /tmp/gh-aw/aw-mcp/logs/. All 20 sampled runs were firewall-enabled and MCP-enabled, and every one of them retained the required debugging artifacts: access.log for the AWF firewall and MCP telemetry via rpc-messages.jsonl.

There were no critical observability gaps in the sampled runs. Coverage was 100% for both firewall access logs and MCP telemetry presence. The main issue is log quality rather than log absence: 19 of 20 firewall-enabled runs did not show both allowed and blocked requests in access.log, so blocked-path visibility remains under-exercised in most successful runs. MCP telemetry was consistently present, but it came entirely from the canonical fallback rpc-messages.jsonl; the preferred gateway.jsonl format was absent from this sample.

Across the sample, the firewall logs contained 3587 entries with 91 blocked requests (2.5%) across 23 unique domains. MCP telemetry contained 298 JSONL events, 105 outgoing tools/call requests, and 8 error/difc-filtered events (2.7%).

Key Alerts and Anomalies

No critical issues detected.

🔴 Critical Issues:

• None.

⚠️ Warnings:

• access.log was present in all 20 sampled firewall-enabled runs, but only 1 run showed both allowed and blocked traffic. 18 runs showed only allowed traffic, and Daily BYOK Ollama Test run 32076040028 showed only blocked traffic.
• MCP telemetry was present in all 20 sampled MCP-enabled runs, but every run relied on the canonical fallback rpc-messages.jsonl; the preferred gateway.jsonl format did not appear in this sample, so duration metrics were not available from the richer source.
• safeoutputs.jsonl was present in 17/20 runs; the missing 3 samples did not affect firewall or MCP coverage, but they reduce structured safe-output traceability.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	20	20	100.0%	✅
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	20	20	100.0%	✅
Overall required observability coverage	20	20	100.0%	✅

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Ponytail Reviewer	32071942913	✅	79	38	0	⚠️ Warning
Matt Pocock Skills Reviewer	32071943766	✅	88	44	0	⚠️ Warning
PR Code Quality Reviewer	32071943035	✅	73	46	0	⚠️ Warning
Duplicate Code Detector	32073363134	✅	199	104	0	⚠️ Warning
ESLint Monster	32073791356	✅	84	52	0	⚠️ Warning
Daily AWF Spec Compiler Surfacing Review	32075282088	✅	42	2	0	⚠️ Warning
Daily BYOK Ollama Test	32076040028	✅	162	0	59	⚠️ Warning
Daily Go Test Parallelizer	32076675381	✅	409	280	0	⚠️ Warning
Daily Reliability Review	32078271580	✅	125	80	0	⚠️ Warning
Matt Pocock Skills Reviewer	32079302526	✅	63	34	0	⚠️ Warning
Ponytail Reviewer	32079302310	✅	58	34	0	⚠️ Warning
Detection Analysis Report	32079411383	✅	547	261	0	⚠️ Warning
Agent Job Health Monitor	32079479453	✅	897	234	0	⚠️ Warning
PR Sous Chef	32079627463	✅	38	20	0	⚠️ Warning
Squad	32080061916	✅	95	36	0	⚠️ Warning
Smoke Copilot	32080621476	✅	207	83	32	✅ Healthy
PR Sous Chef	32080816855	✅	84	49	0	⚠️ Warning
Avenger	32080927705	✅	56	33	0	⚠️ Warning
Squad	32080975933	✅	140	54	0	⚠️ Warning
Cache directory setup	32081032093	✅	141	74	0	⚠️ Warning

Missing Firewall Logs (access.log)

None in the sampled runs.

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Ponytail Reviewer	32071942913	rpc-messages.jsonl	10	github, safeoutputs	2	1	✅ Healthy
Matt Pocock Skills Reviewer	32071943766	rpc-messages.jsonl	4	safeoutputs	1	0	✅ Healthy
PR Code Quality Reviewer	32071943035	rpc-messages.jsonl	4	safeoutputs	1	0	✅ Healthy
Duplicate Code Detector	32073363134	rpc-messages.jsonl	50	github, safeoutputs, serena	20	0	✅ Healthy
ESLint Monster	32073791356	rpc-messages.jsonl	16	safeoutputs	7	0	✅ Healthy
Daily AWF Spec Compiler Surfacing Review	32075282088	rpc-messages.jsonl	2	safeoutputs	0	0	✅ Healthy
Daily BYOK Ollama Test	32076040028	rpc-messages.jsonl	6	github, safeoutputs	0	0	✅ Healthy
Daily Go Test Parallelizer	32076675381	rpc-messages.jsonl	8	github, safeoutputs	1	0	✅ Healthy
Daily Reliability Review	32078271580	rpc-messages.jsonl	8	github, safeoutputs	1	0	✅ Healthy
Matt Pocock Skills Reviewer	32079302526	rpc-messages.jsonl	4	safeoutputs	1	0	✅ Healthy
Ponytail Reviewer	32079302310	rpc-messages.jsonl	10	github, safeoutputs	2	1	✅ Healthy
Detection Analysis Report	32079411383	rpc-messages.jsonl	42	agenticworkflows, github, safeoutputs	17	0	✅ Healthy
Agent Job Health Monitor	32079479453	rpc-messages.jsonl	18	agenticworkflows, safeoutputs	7	0	✅ Healthy
PR Sous Chef	32079627463	rpc-messages.jsonl	4	safeoutputs	1	0	✅ Healthy
Squad	32080061916	rpc-messages.jsonl	20	safeoutputs	9	0	✅ Healthy
Smoke Copilot	32080621476	rpc-messages.jsonl	44	agenticworkflows, mcpscripts, safeoutputs, serena	17	2	✅ Healthy
PR Sous Chef	32080816855	rpc-messages.jsonl	28	safeoutputs	13	4	✅ Healthy
Avenger	32080927705	rpc-messages.jsonl	4	safeoutputs	1	0	✅ Healthy
Squad	32080975933	rpc-messages.jsonl	8	safeoutputs	3	0	✅ Healthy
Cache directory setup	32081032093	rpc-messages.jsonl	8	github, safeoutputs	1	0	✅ Healthy

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

None in the sampled runs.

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 3587
• Domains accessed: 23 unique
• Blocked requests: 91 (2.5%)
• Most accessed domains: api.githubcopilot.com:443, api.anthropic.com:443, o205451.ingest.us.sentry.io:443, otlp-gateway-prod-eu-west-2.grafana.net:443, api.openai.com:443
• Runs showing both allowed and blocked firewall traffic: 1/20 (Smoke Copilot run 32080621476)

Gateway Log Quality

• Telemetry source coverage: rpc-messages.jsonl (20)
• Total entries analyzed: 298
• MCP servers used: safeoutputs, github, agenticworkflows, serena, mcpscripts
• Total tool calls: 105
• Error rate: 2.7% of telemetry events
• Average response time: N/A in this sample because telemetry came from rpc-messages.jsonl rather than gateway.jsonl

Additional Telemetry Coverage

• agent-stdio.log present: 20/20
• Agent output artifact present: 20/20
• awf-config.json or aw_info.json present: 20/20
• safeoutputs.jsonl present: 17/20

Healthy Runs Summary

• Smoke Copilot run 32080621476 had complete firewall and MCP coverage with both allowed and blocked firewall traffic visible.

Recommended Actions

1. Add an explicit blocked-egress probe to at least one daily smoke workflow so successful runs routinely prove that denied traffic is captured in access.log, not just allowed traffic.
2. Emit gateway.jsonl alongside rpc-messages.jsonl in the MCP gateway path so duration and status metrics are available without reconstructing them from raw RPC traffic.
3. Standardize safeoutputs.jsonl publication across all agentic runs so structured output traces are consistently available during postmortems.

📊 Historical Trends

Historical trend data was not loaded for this run. This report is a point-in-time sample of the last 7 days, capped at 20 completed runs.

---

Report generated automatically by the Daily Observability Report workflow
Analysis window: Last 7 days | Runs analyzed: 20

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · gpt54 · 87.3 AIC · ⌖ 11.2 AIC · ⊞ 14.2K · ◷

• expires on Aug 18, 2026, 3:56 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #53859.