[observability] Observability Coverage Report - 2026-08-31

[github-actions[bot]]

Executive Summary

The capped broad logs fetch for the last 7 days returned 30 recent runs before hitting the count limit, spanning 2026-08-31T21:18:26Z through 2026-08-31T23:40:26Z. From that pool, I analyzed 18 runs while keeping at most 2 runs per workflow and prioritizing failures first. Two sampled runs (Detection Analysis Report and Agent Job Health Monitor) were not firewall- or MCP-enabled, so the component coverage denominators below are 16 runs for AWF Firewall and 16 runs for MCP.

No critical observability gaps were found in the enabled sample. Every firewall-enabled run had a discovered access.log at run-<id>/sandbox/firewall/logs/access.log, and every MCP-enabled run had discovered telemetry at run-<id>/mcp-logs/rpc-messages.jsonl. The main weakness is quality depth on firewall evidence rather than coverage: only 4 of 16 firewall-enabled runs showed both allowed and blocked traffic, while the other 12 showed allowed traffic only. MCP telemetry was materially stronger: 16 of 16 enabled runs had valid JSONL with the required top-level fields and zero recorded protocol errors.

Key Alerts and Anomalies

Note

No critical issues detected. No sampled firewall-enabled run was missing access.log, and no sampled MCP-enabled run was missing both gateway.jsonl and rpc-messages.jsonl.

Warning

Firewall observability quality is weaker than MCP observability quality. Only 4 of 16 firewall-enabled runs showed both allow and deny events in access.log, so 12 runs were downgraded to Warning even though the file existed.

Warning

All sampled MCP-enabled runs relied on the canonical fallback rpc-messages.jsonl; no sampled run emitted gateway.jsonl. Coverage is complete, but the preferred structured gateway log was absent in this sample, which leaves duration metrics unavailable without request/response pairing.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	16	16	100%	Warning
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	16	16	100%	Healthy

Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Agentic Workflow Audit Agent	§33440666638	Present	10	4	0	Warning
Daily Regulatory Report Generator	§33440737414	Present	10	4	0	Warning
Duplicate Code Detector	§33443253855	Present	13	6	0	Warning
Daily BYOK Ollama Test	§33446094042	Present	4	2	0	Warning
Daily Go Test Parallelizer	§33446425842	Present	10	4	0	Warning
PR Sous Chef	§33440704982	Present	35	16	0	Warning
PR Sous Chef	§33443759168	Present	60	34	1	Healthy
Auto-Triage Issues	§33441105868	Present	28	16	0	Warning
Auto-Triage Issues	§33441582113	Present	30	16	0	Warning
Issue Monster	§33442430358	Present	45	16	1	Healthy
Issue Monster	§33445642840	Present	39	16	1	Healthy
ESLint Monster	§33443708611	Present	29	14	0	Warning
[aw] Failure Investigator (6h)	§33444348211	Present	40	24	1	Healthy
Daily AWF Spec Compiler Surfacing Review	§33445772684	Present	78	50	0	Warning
Daily Reliability Review	§33448216996	Present	72	53	0	Warning
Cache directory setup	§33450867235	Present	135	70	0	Warning

Missing Firewall Logs (access.log)

None in the sampled firewall-enabled runs.

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Agentic Workflow Audit Agent	§33440666638	rpc-messages.jsonl	8	3	0	0	Healthy
Daily Regulatory Report Generator	§33440737414	rpc-messages.jsonl	8	3	0	0	Healthy
Duplicate Code Detector	§33443253855	rpc-messages.jsonl	10	3	0	0	Healthy
Daily BYOK Ollama Test	§33446094042	rpc-messages.jsonl	6	2	0	0	Healthy
Daily Go Test Parallelizer	§33446425842	rpc-messages.jsonl	6	2	0	0	Healthy
PR Sous Chef	§33440704982	rpc-messages.jsonl	4	1	1	0	Healthy
PR Sous Chef	§33443759168	rpc-messages.jsonl	6	1	2	0	Healthy
Auto-Triage Issues	§33441105868	rpc-messages.jsonl	4	1	1	0	Healthy
Auto-Triage Issues	§33441582113	rpc-messages.jsonl	4	1	1	0	Healthy
Issue Monster	§33442430358	rpc-messages.jsonl	14	1	6	0	Healthy
Issue Monster	§33445642840	rpc-messages.jsonl	14	1	6	0	Healthy
ESLint Monster	§33443708611	rpc-messages.jsonl	4	1	1	0	Healthy
[aw] Failure Investigator (6h)	§33444348211	rpc-messages.jsonl	8	2	2	0	Healthy
Daily AWF Spec Compiler Surfacing Review	§33445772684	rpc-messages.jsonl	4	1	1	0	Healthy
Daily Reliability Review	§33448216996	rpc-messages.jsonl	8	2	1	0	Healthy
Cache directory setup	§33450867235	rpc-messages.jsonl	8	2	1	0	Healthy

Missing MCP Telemetry

None in the sampled MCP-enabled runs.

Non-Applicable Sampled Runs

Workflow	Run ID	Firewall	MCP
Detection Analysis Report	§33450075203	N/A	N/A
Agent Job Health Monitor	§33450107133	N/A	N/A

Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 638
• Runs with both allow and deny evidence: 4 of 16
• Classified allow/deny decisions: 349 allow or deny events, including 4 blocked requests (1.1% blocked)
• Global discovered domains: 11 unique domain:port values
• Most accessed domains: api.openai.com:443 (122), registry.npmjs.org:443 (62), api.githubcopilot.com:443 (58)
• Exact discovered path pattern: all sampled firewall-enabled runs resolved to run-<id>/sandbox/firewall/logs/access.log

Gateway Log Quality

• Telemetry source across the enabled sample: 16 of 16 runs used rpc-messages.jsonl; 0 of 16 runs emitted gateway.jsonl
• Total JSONL entries analyzed: 116
• MCP servers observed: safeoutputs (78 events), github (28), serena (4), agenticworkflows (4), mcpscripts (2)
• Total outgoing tools/call requests: 23, all attributed to safeoutputs
• Error rate: 0% observed protocol errors or RPC error payloads in the sampled MCP telemetry
• Average response time: N/A in this report because the sample relied on rpc-messages.jsonl fallback rather than gateway.jsonl duration fields
• Exact discovered path pattern: all sampled MCP-enabled runs resolved to run-<id>/mcp-logs/rpc-messages.jsonl

Additional Telemetry Coverage

• agent-stdio.log present in 16 of 16 component-enabled runs
• awf-config.json present in 16 of 16 component-enabled runs
• Structured safe-output JSONL present in 11 of 18 sampled runs
• aw_info.json present in 0 of 18 sampled runs; available configuration metadata came from awf-config.json instead

Healthy Runs Summary

The following sampled runs met the strict Healthy criteria for both firewall and MCP telemetry quality:

• §33443759168 PR Sous Chef
• §33442430358 Issue Monster
• §33445642840 Issue Monster
• §33444348211 [aw] Failure Investigator (6h)

Recommended Actions

1. Emit gateway.jsonl alongside rpc-messages.jsonl for MCP-enabled runs so duration, status, and per-call metrics are available without fallback pairing.
2. Add a lightweight deny-path exercise or synthetic blocked request check in firewall-enabled workflows so access.log routinely captures both allow and block behavior.
3. Standardize secondary observability artifacts across workflows, especially structured safe-output JSONL and a single canonical metadata file name (aw_info.json or awf-config.json).

Historical Trends

Trend analysis is not included in this daily report. The required -7d broad fetch hit the 30-run cap after roughly two hours of activity, so this report reflects a representative capped sample from the returned pool rather than a full week trendline.

References: §33443759168, §33440666638, §33450867235

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · codex · gpt54 · 99.4 AIC · ⌖ 11.2 AIC · ⊞ 14.4K · ◷

• expires on Sep 1, 2026, 4:02 PM UTC-08:00