[observability] Observability Coverage Report - 2026-04-12

[github-actions[bot]]

Executive Summary

Observability artifacts were analyzed for the last 7 days using locally downloaded workflow-run logs under /tmp/gh-aw/aw-mcp/logs and filtered by run creation time (>= 2026-04-05). A total of 229 runs had readable run_summary.json metadata in this window.

The audit found a major telemetry gap for both target components. For runs flagged as firewall-enabled by aw_info.json, 0/121 had readable access.log artifacts. For runs with MCP log directories, only 24/120 had MCP telemetry (rpc-messages.jsonl), and 0 had gateway.jsonl.

Key Alerts and Anomalies

🔴 Critical Issues:

• Firewall gap: access.log missing in 121/121 firewall-enabled runs (100% missing).
• MCP gap: Both gateway.jsonl and rpc-messages.jsonl absent in 96/120 MCP-enabled runs (80% missing).

⚠️ Warnings:

• rpc-messages.jsonl present but empty in 1 run: §24164515813.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	121	0	0.0%	🔴
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	120	24	20.0%	🔴

Additional context:

• Runs analyzed in window: 229
• Runs with complete required coverage under this audit logic: 106/229

📋 Detailed Run Analysis

Firewall-Enabled Runs (Most Recent 20)

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Workflow Normalizer	§24319384124	❌	0	0	0	🔴
Changeset Generator	§24319321826	❌	0	0	0	🔴
Smoke Claude	§24319321832	❌	0	0	0	🔴
Smoke Codex	§24319321846	❌	0	0	0	🔴
Smoke Copilot	§24319321848	❌	0	0	0	🔴
Agent Container Smoke Test	§24319321853	❌	0	0	0	🔴
Daily Observability Report for AWF Firewall and MCP Gateway	§24319274953	❌	0	0	0	🔴
Daily Semgrep Scan	§24319251860	❌	0	0	0	🔴
Auto-Triage Issues	§24319171263	❌	0	0	0	🔴
AI Moderator	§24319062208	❌	0	0	0	🔴
GitHub API Consumption Report Agent	§24319042576	❌	0	0	0	🔴
Daily Project Performance Summary Generator (Using MCP Scripts)	§24318898202	❌	0	0	0	🔴
Daily Safe Output Integrator	§24318496095	❌	0	0	0	🔴
AI Moderator	§24318017770	❌	0	0	0	🔴
Daily Secrets Analysis Agent	§24317705014	❌	0	0	0	🔴
Draft PR Cleanup	§24317403302	❌	0	0	0	🔴
Daily Safe Outputs Conformance Checker	§24317377277	❌	0	0	0	🔴
Design Decision Gate 🏗️	§24317277255	❌	0	0	0	🔴
Test Quality Sentinel	§24317277256	❌	0	0	0	🔴
Smoke Service Ports	§24317089887	❌	0	0	0	🔴

Missing Firewall Logs (access.log) (Most Recent 20)

Workflow	Run ID	Date	Link
Workflow Normalizer	24319384124	2026-04-12	§24319384124
Changeset Generator	24319321826	2026-04-12	§24319321826
Smoke Claude	24319321832	2026-04-12	§24319321832
Smoke Codex	24319321846	2026-04-12	§24319321846
Smoke Copilot	24319321848	2026-04-12	§24319321848
Agent Container Smoke Test	24319321853	2026-04-12	§24319321853
Daily Observability Report for AWF Firewall and MCP Gateway	24319274953	2026-04-12	§24319274953
Daily Semgrep Scan	24319251860	2026-04-12	§24319251860
Auto-Triage Issues	24319171263	2026-04-12	§24319171263
AI Moderator	24319062208	2026-04-12	§24319062208
GitHub API Consumption Report Agent	24319042576	2026-04-12	§24319042576
Daily Project Performance Summary Generator (Using MCP Scripts)	24318898202	2026-04-12	§24318898202
Daily Safe Output Integrator	24318496095	2026-04-12	§24318496095
AI Moderator	24318017770	2026-04-12	§24318017770
Daily Secrets Analysis Agent	24317705014	2026-04-12	§24317705014
Draft PR Cleanup	24317403302	2026-04-12	§24317403302
Daily Safe Outputs Conformance Checker	24317377277	2026-04-12	§24317377277
Design Decision Gate 🏗️	24317277255	2026-04-12	§24317277255
Test Quality Sentinel	24317277256	2026-04-12	§24317277256
Smoke Service Ports	24317089887	2026-04-12	§24317089887

MCP-Enabled Runs (Most Recent 20)

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Smoke Gemini	§24319321835	❌ None	0	0	0	0	🔴
Smoke Codex	§24319321846	❌ None	0	0	0	0	🔴
Agent Container Smoke Test	§24319321853	❌ None	0	0	0	0	🔴
Daily Semgrep Scan	§24319251860	❌ None	0	0	0	0	🔴
Auto-Triage Issues	§24319171263	❌ None	0	0	0	0	🔴
AI Moderator	§24319062208	❌ None	0	0	0	0	🔴
GitHub API Consumption Report Agent	§24319042576	rpc-messages.jsonl	22	3	8	0	✅
Daily Safe Output Integrator	§24318496095	❌ None	0	0	0	0	🔴
AI Moderator	§24318017770	❌ None	0	0	0	0	🔴
Daily Secrets Analysis Agent	§24317705014	❌ None	0	0	0	0	🔴
Draft PR Cleanup	§24317403302	❌ None	0	0	0	0	🔴
Daily Safe Outputs Conformance Checker	§24317377277	❌ None	0	0	0	0	🔴
Design Decision Gate 🏗️	§24317277255	❌ None	0	0	0	0	🔴
Test Quality Sentinel	§24317277256	❌ None	0	0	0	0	🔴
Smoke Service Ports	§24317089887	❌ None	0	0	0	0	🔴
Daily Compiler Quality Check	§24316997240	❌ None	0	0	0	0	🔴
Daily Code Metrics and Trend Tracking Agent	§24316849082	❌ None	0	0	0	0	🔴
Daily Malicious Code Scan Agent	§24316650614	❌ None	0	0	0	0	🔴
Daily OTel Instrumentation Advisor	§24316622621	❌ None	0	0	0	0	🔴
Go Logger Enhancement	§24316620973	❌ None	0	0	0	0	🔴

Missing MCP Telemetry (Most Recent 20)

Workflow	Run ID	Date	Link
Smoke Gemini	24319321835	2026-04-12	§24319321835
Smoke Codex	24319321846	2026-04-12	§24319321846
Agent Container Smoke Test	24319321853	2026-04-12	§24319321853
Daily Semgrep Scan	24319251860	2026-04-12	§24319251860
Auto-Triage Issues	24319171263	2026-04-12	§24319171263
AI Moderator	24319062208	2026-04-12	§24319062208
Daily Safe Output Integrator	24318496095	2026-04-12	§24318496095
AI Moderator	24318017770	2026-04-12	§24318017770
Daily Secrets Analysis Agent	24317705014	2026-04-12	§24317705014
Draft PR Cleanup	24317403302	2026-04-12	§24317403302
Daily Safe Outputs Conformance Checker	24317377277	2026-04-12	§24317377277
Design Decision Gate 🏗️	24317277255	2026-04-12	§24317277255
Test Quality Sentinel	24317277256	2026-04-12	§24317277256
Smoke Service Ports	24317089887	2026-04-12	§24317089887
Daily Compiler Quality Check	24316997240	2026-04-12	§24316997240
Daily Code Metrics and Trend Tracking Agent	24316849082	2026-04-12	§24316849082
Daily Malicious Code Scan Agent	24316650614	2026-04-12	§24316650614
Daily OTel Instrumentation Advisor	24316622621	2026-04-12	§24316622621
Go Logger Enhancement	24316620973	2026-04-12	§24316620973

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 0
• Domains accessed: 0
• Blocked requests: 0
• Most accessed domains: N/A (no readable firewall logs in the analyzed set)

Gateway/MCP Log Quality

• Telemetry source distribution: gateway.jsonl: 0, rpc-messages.jsonl: 24, none: 96
• Total MCP telemetry entries analyzed: 472
• MCP servers observed in telemetry: agenticworkflows, github, mcpscripts, playwright, safeoutputs, serena, tavily
• Total tool calls from MCP telemetry: 158
• Error rate: 0.0% (0 errors / 472 entries)
• Average response time: N/A (duration not consistently available from rpc fallback entries)

Healthy Runs Summary

• MCP telemetry healthy runs: 24
• Firewall telemetry healthy runs: 0
• One run with telemetry-quality warning (0 entries despite file presence): §24164515813

Recommended Actions

1. Make access.log artifact upload mandatory for all runs with firewall_enabled=true, and fail the run if artifact collection is skipped.
2. Enforce MCP telemetry invariant: if mcp-logs/ exists, require at least one of gateway.jsonl or rpc-messages.jsonl; fail post-job validation otherwise.
3. Add a lightweight observability conformance check step in every workflow to emit explicit pass/fail for firewall-log presence and MCP-log presence before safe output publication.

📊 Historical Trends

• 2026-04-06 to 2026-04-11: only one analyzed run/day in the local dataset (daily report runs), MCP coverage 100% on those runs, no firewall-enabled runs identified via aw_info.
• 2026-04-12: 223 analyzed runs, firewall coverage 0.0% (0/121), MCP coverage 15.8% (18/114 for that day).

Context

• Analysis window: last 7 days
• Runs analyzed: 229 (runs with readable run_summary.json in local downloaded artifacts)
• Triggering run: §24319274953

References:

• §24319274953
• §24294226379
• §24319384124

Warning

⚠️ Firewall blocked 4 domains

The following domains were blocked by the firewall during workflow execution:

• ab.chatgpt.com
• api.github.com
• chatgpt.com
• github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "api.github.com"
    - "chatgpt.com"
    - "github.com"

See Network Configuration for more information.

Generated by Daily Observability Report for AWF Firewall and MCP Gateway · ◷

• expires on Apr 14, 2026, 12:00 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #26110.