[observability] Observability Coverage Report - 2026-03-25

[github-actions[bot]]

Executive Summary

Over the last 7 days, 19 runs with full agent artifacts were analyzed for observability coverage of AWF Firewall and MCP Gateway telemetry. Firewall coverage is incomplete: 12/19 runs had access.log available, while 4 completed runs were missing firewall access logs entirely and 3 in-progress runs had not produced logs yet.

MCP telemetry coverage is complete for MCP-enabled runs. All 12 MCP-enabled runs had canonical MCP telemetry via mcp-logs/rpc-messages.jsonl (with no runs missing both gateway.jsonl and rpc-messages.jsonl). No MCP-telemetry critical gaps were detected.

Key Alerts and Anomalies

Critical observability gaps were detected in firewall logs for completed runs.

Critical Issues:

• Missing firewall access logs in completed runs: §23566025032, §23566025048, §23568997760, §23568997793.

Warnings:

• 3 in-progress runs currently have no firewall access logs yet: §23569912987, §23569922546, §23570094201.
• 8 completed firewall-log runs had allowed traffic but no blocked entries, reducing policy-verification confidence.
• mcp-logs directories are permission-restricted in downloaded artifacts, so MCP entry-level parsing (event counts/durations) is partially unavailable in this report.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	19	12	63.2%	Critical
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	12	12	100.0%	Healthy

Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Changeset Generator	§23566025018	Yes	57	18	4	Healthy
Smoke Codex	§23566025025	Yes	79	24	5	Healthy
Agent Container Smoke Test	§23566025030	Yes	19	10	0	Warning
Smoke Claude	§23566025032	No	0	0	0	Critical
Smoke Copilot	§23566025048	No	0	0	0	Critical
Daily Safe Outputs Conformance Checker	§23566307796	Yes	13	5	0	Warning
Draft PR Cleanup	§23566356573	Yes	33	17	0	Warning
Daily Secrets Analysis Agent	§23567040912	Yes	47	18	0	Warning
Daily Safe Output Integrator	§23568514668	Yes	31	13	0	Warning
Changeset Generator	§23568997759	Yes	121	55	8	Healthy
Smoke Claude	§23568997760	No	0	0	0	Critical
Smoke Codex	§23568997764	Yes	92	30	5	Healthy
Agent Container Smoke Test	§23568997782	Yes	19	10	0	Warning
Smoke Copilot	§23568997793	No	0	0	0	Critical
Daily Project Performance Summary Generator (Using MCP Scripts)	§23569260216	Yes	51	13	0	Warning
Daily Semgrep Scan	§23569875090	Yes	27	14	0	Warning
Daily Observability Report for AWF Firewall and MCP Gateway	§23569912987	No	0	0	0	Warning
Daily Safe Output Tool Optimizer	§23569922546	No	0	0	0	Warning
Workflow Normalizer	§23570094201	No	0	0	0	Warning

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Link
Smoke Claude	23566025032	2026-03-25	§23566025032
Smoke Copilot	23566025048	2026-03-25	§23566025048
Smoke Claude	23568997760	2026-03-25	§23568997760
Smoke Copilot	23568997793	2026-03-25	§23568997793

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Changeset Generator	§23566025018	rpc-messages.jsonl	N/A	2	1	0	Healthy
Smoke Codex	§23566025025	rpc-messages.jsonl	N/A	7	16	0	Healthy
Agent Container Smoke Test	§23566025030	rpc-messages.jsonl	N/A	2	N/A	0	Healthy
Daily Safe Outputs Conformance Checker	§23566307796	rpc-messages.jsonl	N/A	2	3	0	Healthy
Draft PR Cleanup	§23566356573	rpc-messages.jsonl	N/A	2	N/A	0	Healthy
Daily Secrets Analysis Agent	§23567040912	rpc-messages.jsonl	N/A	2	N/A	0	Healthy
Daily Safe Output Integrator	§23568514668	rpc-messages.jsonl	N/A	2	N/A	0	Healthy
Changeset Generator	§23568997759	rpc-messages.jsonl	N/A	2	2	0	Healthy
Smoke Codex	§23568997764	rpc-messages.jsonl	N/A	7	16	0	Healthy
Agent Container Smoke Test	§23568997782	rpc-messages.jsonl	N/A	2	N/A	0	Healthy
Daily Project Performance Summary Generator (Using MCP Scripts)	§23569260216	rpc-messages.jsonl	N/A	3	N/A	1	Healthy
Daily Semgrep Scan	§23569875090	rpc-messages.jsonl	N/A	3	N/A	0	Healthy

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

Workflow	Run ID	Date	Link
None	-	-	-

Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 589
• Domains accessed: 10 unique
• Blocked requests: 22 (3.7%)
• Most accessed domains: api.openai.com:443, api.githubcopilot.com:443, ab.chatgpt.com:443

Gateway Log Quality

• Telemetry source in MCP-enabled runs: rpc-messages.jsonl (canonical fallback)
• Total MCP-enabled runs: 12
• MCP servers observed per run: typically 2-3 (up to 7 in smoke codex runs)
• Tool call visibility: partial (metrics.ToolCalls available for 5/12 MCP-enabled runs; 38 calls across those 5 runs)
• Error rate in MCP-enabled runs: 1/12 runs with non-zero error count (8.3%)
• Average response time: N/A in this report due restricted direct access to mcp-logs JSONL payloads

Healthy Runs Summary

• Firewall healthy runs (access.log present with both allowed and blocked events): 4/19
• MCP telemetry healthy runs (gateway/rpc present): 12/12 MCP-enabled runs

Recommended Actions

1. Add a CI guard in agent jobs to fail when firewall is enabled but firewall-audit-logs/logs/access.log is missing at artifact upload time.
2. Standardize firewall log publication for smoke workflows (Smoke Claude, Smoke Copilot) so successful runs always include access.log.
3. Export a lightweight MCP telemetry summary artifact (entries, server count, error count, latency percentiles) outside restricted mcp-logs permissions to improve daily observability reporting quality.

Historical Trends

Trend baseline is not included in this run because only the current 7-day window snapshot was processed for this report.

References:

• §23566025032
• §23568997793
• §23569260216

AI generated by Daily Observability Report for AWF Firewall and MCP Gateway · history

• expires on Mar 26, 2026, 11:59 PM UTC