[observability] Observability Coverage Report - 2026-06-18

[github-actions[bot]]

Executive Summary

This week’s 20-run sample is mostly healthy, but not clean. Every analyzed run was firewall-enabled, access.log was retained on 16/20 runs, and MCP telemetry was retained on 17/20 runs. The MCP side is relying entirely on rpc-messages.jsonl; no gateway.jsonl files were observed in this sample.

The main issue is missing firewall telemetry on four runs, including one completed workflow and three in-progress workflows. The three in-progress runs are also missing MCP telemetry, so they are not debuggable from the downloaded artifacts alone.

Key Alerts and Anomalies

🔴 Critical Issues:

• run-27726899573 (Changeset Generator) is missing access.log.
• run-27727716397 (Daily Documentation Healer) is missing access.log and MCP telemetry.
• run-27727741467 (Semantic Function Refactoring) is missing access.log and MCP telemetry.
• run-27727766724 (Daily Observability Report for AWF Firewall and MCP Gateway) is missing access.log and MCP telemetry.

⚠️ Warnings:

• Several firewall logs are dominated by error:transaction-end-before-headers, so the proxy trace is present but noisy.
• Structured gateway timing is unavailable in this sample because telemetry is coming from rpc-messages.jsonl only.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	20	16	80%	⚠️
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	20	17	85%	⚠️

📋 Detailed Run Analysis

Run Summary

Workflow	Run ID	Firewall	MCP Telemetry	Status
Daily Observability Report for AWF Firewall and MCP Gateway	27727766724	❌	❌	🔴
Semantic Function Refactoring	27727741467	❌	❌	🔴
Daily Documentation Healer	27727716397	❌	❌	🔴
Changeset Generator	27726899573	❌	✅ (rpc-messages.jsonl)	🔴
Smoke CI	27727400733	✅	✅ (rpc-messages.jsonl)	✅
Daily Security Red Team Agent	27727399874	✅	✅ (rpc-messages.jsonl)	✅
Smoke CI	27727396007	✅	✅ (rpc-messages.jsonl)	✅
Smoke Copilot - AOAI (Entra)	27726926051	✅	✅ (rpc-messages.jsonl)	✅
Smoke Copilot - AOAI (apikey)	27726925449	✅	✅ (rpc-messages.jsonl)	✅
Smoke Copilot	27726924811	✅	✅ (rpc-messages.jsonl)	✅
Smoke Gemini	27726899637	✅	✅ (rpc-messages.jsonl)	✅
Agent Container Smoke Test	27726899589	✅	✅ (rpc-messages.jsonl)	✅
Smoke Pi	27726899585	✅	✅ (rpc-messages.jsonl)	✅
Smoke Codex	27726899556	✅	✅ (rpc-messages.jsonl)	✅
Smoke Antigravity	27726899551	✅	✅ (rpc-messages.jsonl)	✅
Smoke Claude	27726899549	✅	✅ (rpc-messages.jsonl)	✅
PR Sous Chef	27726614160	✅	✅ (rpc-messages.jsonl)	✅
Auto-Triage Issues	27726126006	✅	✅ (rpc-messages.jsonl)	✅
Auto-Triage Issues	27726123025	✅	✅ (rpc-messages.jsonl)	✅
Daily Reliability Review	27725869956	✅	✅ (rpc-messages.jsonl)	✅

Missing Firewall Logs (access.log)

Workflow	Run ID	Link
Changeset Generator	27726899573	§27726899573
Daily Documentation Healer	27727716397	§27727716397
Semantic Function Refactoring	27727741467	§27727741467
Daily Observability Report for AWF Firewall and MCP Gateway	27727766724	§27727766724

Missing MCP Telemetry

Workflow	Run ID	Link
Daily Documentation Healer	27727716397	§27727716397
Semantic Function Refactoring	27727741467	§27727741467
Daily Observability Report for AWF Firewall and MCP Gateway	27727766724	§27727766724

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 1,816
• Unique destination domains: 29
• Allowed CONNECTs: 517
• Explicit denials: 109
• Proxy/client abort noise: 1,190
• Most accessed domains: api.anthropic.com, api.githubcopilot.com, gh-aw-foundry.openai.azure.com, github.githubassets.com, generativelanguage.googleapis.com
• Most common denials: www.google.com, content-autofill.googleapis.com, localhost:8080, accounts.google.com

MCP Gateway Log Quality

• Telemetry source: rpc-messages.jsonl (canonical fallback)
• Total entries analyzed: 438
• MCP tool calls: 185
• MCP servers used: sentry, safeoutputs, serena, mcpscripts, tavily, agenticworkflows
• Error rate: 0%
• Response-time metrics: not available as structured gateway timing in this sample

Healthy Runs Summary

• The sampled healthy runs retained both firewall and MCP artifacts.
• The strongest coverage is on the Copilot and Sentry-heavy workflows, where both allow and deny events are visible in the Squid trace and RPC activity is fully captured.

Recommended Actions

1. Preserve sandbox/firewall/logs/access.log for every firewall-enabled run, including in-progress daily jobs, so the proxy trace survives long enough for debugging.
2. Keep rpc-messages.jsonl as the minimum MCP fallback, but restore gateway.jsonl where possible to recover per-call timing and error attribution.
3. Reduce or filter transaction-end-before-headers noise in summaries so small firewall logs stay readable when a real deny event appears.

References:

• §27727766724
• §27726899573
• §27727741467

Report generated automatically by the Daily Observability Report workflow
Analysis window: Last 7 days | Runs analyzed: 20

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• api.github.com
• github.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.github.com"
    - "github.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · ◷

• expires on Jun 18, 2026, 4:16 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #40172.