[observability] Observability Coverage Report - 2026-05-12

[github-actions[bot]]

Executive Summary

Analyzed the latest 100 agentic workflow runs from the last 7 days using the agenticworkflows logs targeted firewall/MCP artifact pass. The run set contains 7 firewall-enabled runs identified by aw_info.json; 4 are completed and eligible for final artifact checks, while 3 were still in progress when logs were fetched.

Completed firewall coverage is degraded: 3 of 4 completed firewall-enabled runs have access.log, for 75.0% coverage. Completed MCP telemetry coverage is complete by presence criteria: 3 of 3 MCP-enabled runs have rpc-messages.jsonl present, for 100.0% coverage, but one fallback log is empty and should be investigated as a quality warning.

Key Alerts and Anomalies

🔴 Critical Issues:

• Smoke CI run §25702296438 is completed, has firewall_enabled: true in aw_info.json, and has no access.log artifact. This prevents normal firewall egress debugging for that run.

⚠️ Warnings:

• Auto-Triage Issues run §25703825374 has rpc-messages.jsonl present, but the file has 0 JSON-RPC entries. Presence satisfies MCP telemetry coverage, but the fallback log has no useful protocol detail.
• Three firewall-enabled runs were still in_progress at collection time and had no final firewall artifacts yet: Daily Documentation Healer §25704595533, Daily Observability Report for AWF Firewall and MCP Gateway §25704658440, and Semantic Function Refactoring §25704640289. These were excluded from completed-run coverage percentages.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	4 completed firewall-enabled runs	3	75.0%	🔴 Critical
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	3 completed MCP-enabled runs	3	100.0%	⚠️ Warning
Combined observability obligations	7 completed component checks	6	85.7%	⚠️ Degraded

Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	Status	access.log	Real Egress Entries	Allowed	Blocked	Assessment
Daily Security Red Team Agent	§25704013750	completed	✅	17	17	0	✅ Healthy
Auto-Triage Issues	§25703825374	completed	✅	6	6	0	✅ Healthy
Smoke CI	§25702354483	completed	✅	2	2	0	✅ Healthy
Smoke CI	§25702296438	completed	❌	0	0	0	🔴 Critical
Daily Documentation Healer	§25704595533	in_progress	pending	0	0	0	i️ Pending
Daily Observability Report for AWF Firewall and MCP Gateway	§25704658440	in_progress	pending	0	0	0	i️ Pending
Semantic Function Refactoring	§25704640289	in_progress	pending	0	0	0	i️ Pending

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Link
Smoke CI	25702296438	2026-05-11	§25702296438

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Assessment
Daily Security Red Team Agent	§25704013750	rpc-messages.jsonl	4	safeoutputs	1	0	✅ Healthy
Auto-Triage Issues	§25703825374	rpc-messages.jsonl	0	none observed	0	0	⚠️ Warning
Smoke CI	§25702354483	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ Healthy

Missing MCP Telemetry

No completed MCP-enabled runs were missing both gateway.jsonl and rpc-messages.jsonl.

Telemetry Quality Analysis

Firewall Log Quality

• Total real egress entries analyzed: 25
• Unique accessed domains: 3
• Allowed requests: 25
• Blocked requests: 0
• Most accessed domains: api.anthropic.com:443 (15), api.githubcopilot.com:443 (8), github.com:443 (2)
• Note: raw Squid logs also contain loopback transaction-end-before-headers noise; the egress counts above use real CONNECT entries.

Gateway Log Quality

• Telemetry source observed: rpc-messages.jsonl canonical fallback
• Preferred gateway.jsonl files observed: 0
• Total JSON-RPC entries analyzed: 10
• MCP servers observed: safeoutputs, github
• Total tool calls observed: 2
• Errors observed: 0
• Average response time: N/A from this pass; no gateway.jsonl duration metrics were present, and response pairing was not available for the empty fallback log.

Data Collection Notes

• The first broad 100-run all-artifact fetch timed out at the bridge layer, but it wrote partial artifacts locally.
• A second targeted 100-run pass for firewall and MCP artifacts completed successfully and is the basis for this report.
• Runs still in progress were listed separately because final artifacts may not have been uploaded yet.

Recommended Actions

1. Investigate why completed Smoke CI run §25702296438 has firewall_enabled: true but no access.log artifact. Confirm whether the firewall teardown/upload step skipped, failed, or ran before Squid created the file.
2. Inspect the MCP gateway/fallback capture path for Auto-Triage Issues run §25703825374, where rpc-messages.jsonl exists but is empty despite MCP logging artifacts being uploaded.
3. Prefer uploading gateway.jsonl in addition to rpc-messages.jsonl so future reports can include duration, status, and error-rate metrics without relying only on JSON-RPC fallback parsing.

Historical Trends

No prior daily trend data was available in the downloaded artifact set. This report should be treated as the baseline for subsequent observability coverage comparisons.

References: §25702296438, §25704013750, §25703825374

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• api.github.com
• github.com

💡 Tip: api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.github.com"
    - "github.com"

See Network Configuration for more information.

Generated by Daily Observability Report for AWF Firewall and MCP Gateway · ◷

• expires on May 13, 2026, 12:13 AM UTC