Doc: document `firewall-audit-logs` artifact name for downstream consumers by Copilot · Pull Request #25684 · github/gh-aw

Copilot · 2026-04-10T17:56:28Z

Downstream repos writing custom token analysis workflows with gh run download had no way to discover that token usage data lives in the firewall-audit-logs artifact, not agent-artifacts or agent. The failure mode is silent — data simply missing with no error.

Changes

New docs/src/content/docs/reference/artifacts.md — Complete artifact reference: all artifact names with Go constants, artifact sets for --artifacts filtering, firewall-audit-logs directory structure, naming compatibility, workflow_call prefix handling, and legacy artifact markers for safe-output/agent-output
Updated scratchpad/artifact-naming-compatibility.md — Split compatibility matrix into single-file vs multi-file sections (with detection correctly categorized as single-file); added firewall-audit-logs structure, download examples, common mistake callout, and clarified CLI post-processing behavior for multi-file artifacts
Updated docs/src/content/docs/reference/compilation-process.md — Added firewall-audit-logs to the "Artifacts Created" table with correct directory structure, fixed firewall-logs/ runtime path, and added cross-link to the new reference page

Key documentation addition

# ❌ WRONG — token-usage.jsonl is NOT in the agent artifact
gh run download <run-id> -n agent

# ✅ CORRECT — download from firewall-audit-logs
gh run download <run-id> -n firewall-audit-logs
cat firewall-audit-logs/api-proxy-logs/token-usage.jsonl

# ✅ RECOMMENDED — let the CLI handle artifact naming
gh aw logs <run-id> --artifacts firewall

Add firewall-audit-logs artifact documentation to help downstream repositories that write custom token analysis workflows using gh run download directly. Changes: - Create docs/src/content/docs/reference/artifacts.md with complete artifact naming reference including directory structures - Update scratchpad/artifact-naming-compatibility.md to add multi-file artifacts section with firewall-audit-logs details - Update docs/src/content/docs/reference/compilation-process.md to include firewall-audit-logs in the Artifacts Created table Closes #1494 Agent-Logs-Url: https://github.com/github/gh-aw/sessions/28093af6-8cb7-4db7-a198-a7f06c6f20e3 Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot

Pull request overview

Adds a dedicated documentation reference so downstream consumers can correctly discover and download token usage / firewall observability data from the firewall-audit-logs artifact (rather than mistakenly pulling agent/agent-artifacts and getting silent omissions).

Changes:

Introduces a new “Artifacts” reference page covering artifact names, sets for --artifacts, compatibility notes, and workflow_call prefix handling.
Expands the artifact naming compatibility scratchpad with single-file vs multi-file sections and firewall artifact guidance.
Updates the compilation process reference to include firewall-audit-logs and document its structure.

Show a summary per file

File	Description
scratchpad/artifact-naming-compatibility.md	Splits compatibility matrix and adds firewall artifact structure + download guidance
docs/src/content/docs/reference/compilation-process.md	Adds `firewall-audit-logs` to artifacts list and documents its structure
docs/src/content/docs/reference/artifacts.md	New comprehensive artifact reference for downstream consumers

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (4)

scratchpad/artifact-naming-compatibility.md:82

The “data is then at” examples use firewall-audit-logs/logs/..., but the CLI/tests expect firewall-audit-logs/api-proxy-logs/token-usage.jsonl and related paths without the logs/ prefix. Adjust these example paths to match the real extracted layout.

# The data is then at:
#   firewall-audit-logs/logs/api-proxy-logs/token-usage.jsonl
#   firewall-audit-logs/logs/squid-logs/access.log
#   firewall-audit-logs/audit/audit.jsonl

**docs/src/content/docs/reference/compilation-process.md:273**
* This structure block only lists `audit/audit.jsonl`, but firewall policy analysis also relies on `policy-manifest.json` (pkg/cli/firewall_policy.go). Consider including `policy-manifest.json` here and ensure the documented path matches the actual extracted layout (the CLI looks for these files at `firewall-audit-logs/policy-manifest.json` and `firewall-audit-logs/audit.jsonl`).

└── audit/
└── audit.jsonl ← Firewall audit trail

docs/src/content/docs/reference/artifacts.md:89

These example paths include firewall-audit-logs/logs/..., but the CLI/tests expect firewall-audit-logs/api-proxy-logs/token-usage.jsonl (no logs/ prefix). Please update the cat example accordingly.

gh run download <run-id> -n firewall-audit-logs

# Token usage data is at:
cat firewall-audit-logs/logs/api-proxy-logs/token-usage.jsonl

docs/src/content/docs/reference/artifacts.md:110

The “correct” example uses firewall-audit-logs/logs/api-proxy-logs/token-usage.jsonl, but the extracted layout used by the CLI/tests is firewall-audit-logs/api-proxy-logs/token-usage.jsonl. Please update this snippet so the copy/paste path resolves.

# ✅ CORRECT — download from firewall-audit-logs
gh run download <run-id> -n firewall-audit-logs
cat firewall-audit-logs/logs/api-proxy-logs/token-usage.jsonl

</details>


- **Files reviewed:** 3/3 changed files
- **Comments generated:** 11

Copilot · 2026-04-10T18:12:16Z

scratchpad/artifact-naming-compatibility.md

+| `agent` | `constants.AgentArtifactName` | Unified agent job outputs (logs, safe outputs, token usage) | The primary agent artifact |
+| `activation` | `constants.ActivationArtifactName` | Activation job output (`aw_info.json`, `prompt.txt`) | Used by downstream jobs |
+| `detection` | `constants.DetectionArtifactName` | Threat detection log output | Legacy name: `threat-detection.log` |


The multi-file artifact list includes detection, but detection is a single-file artifact (it contains only detection.log) and is flattened by flattenSingleFileArtifacts(). Consider moving detection into the Single-File Artifacts matrix (old threat-detection.log → new detection) to avoid implying it retains a directory tree.

Copilot · 2026-04-10T18:12:16Z

scratchpad/artifact-naming-compatibility.md

+These artifacts contain a directory tree and are **not** flattened. They retain their directory structure after download.
+
+| Artifact Name | Constant | Contents | Notes |
+|---------------|----------|----------|-------|
+| `firewall-audit-logs` | `constants.FirewallAuditArtifactName` | AWF structured audit/observability logs | Uploaded by all firewall-enabled workflows |
+| `agent` | `constants.AgentArtifactName` | Unified agent job outputs (logs, safe outputs, token usage) | The primary agent artifact |
+| `activation` | `constants.ActivationArtifactName` | Activation job output (`aw_info.json`, `prompt.txt`) | Used by downstream jobs |
+| `detection` | `constants.DetectionArtifactName` | Threat detection log output | Legacy name: `threat-detection.log` |


This section says multi-file artifacts are “not flattened” and “retain their directory structure after download,” but the CLI does flatten at least activation and agent via flattenActivationArtifact() / flattenUnifiedArtifact() (pkg/cli/logs_download.go). Please clarify whether this statement is meant for raw gh run download output vs the post-processed layout produced by gh aw logs/gh aw audit.

Suggested change

These artifacts contain a directory tree and are **not** flattened. They retain their directory structure after download.

| Artifact Name | Constant | Contents | Notes |

|---------------|----------|----------|-------|

| `firewall-audit-logs` | `constants.FirewallAuditArtifactName` | AWF structured audit/observability logs | Uploaded by all firewall-enabled workflows |

| `agent` | `constants.AgentArtifactName` | Unified agent job outputs (logs, safe outputs, token usage) | The primary agent artifact |

| `activation` | `constants.ActivationArtifactName` | Activation job output (`aw_info.json`, `prompt.txt`) | Used by downstream jobs |

| `detection` | `constants.DetectionArtifactName` | Threat detection log output | Legacy name: `threat-detection.log` |

These artifacts are initially downloaded by `gh run download` as directory trees that retain their internal structure. However, unlike the single-file artifact handling above, `gh aw logs` / `gh aw audit` may perform additional post-processing for some multi-file artifacts (notably `agent` and `activation`) to move expected files into the final layout used by the CLI.

| Artifact Name | Constant | Contents | Notes |

|---------------|----------|----------|-------|

| `firewall-audit-logs` | `constants.FirewallAuditArtifactName` | AWF structured audit/observability logs | Uploaded by all firewall-enabled workflows; retains directory structure after download |

| `agent` | `constants.AgentArtifactName` | Unified agent job outputs (logs, safe outputs, token usage) | Downloaded as a directory tree, then post-processed by CLI flattening/reorganization helpers |

| `activation` | `constants.ActivationArtifactName` | Activation job output (`aw_info.json`, `prompt.txt`) | Downloaded as a directory tree, then post-processed by CLI flattening helpers for downstream use |

| `detection` | `constants.DetectionArtifactName` | Threat detection log output | Legacy name: `threat-detection.log`; remains as downloaded unless separately processed |

Copilot · 2026-04-10T18:12:17Z

scratchpad/artifact-naming-compatibility.md

+└── audit/
+    ├── audit.jsonl              ← Firewall audit trail (policy matches, rule evaluations)
+    └── policy-manifest.json     ← Policy configuration snapshot


The firewall audit files are shown under firewall-audit-logs/audit/…, but the CLI’s firewall policy loader looks for policy-manifest.json and audit.jsonl at the root of the firewall-audit-logs directory (pkg/cli/firewall_policy.go + pkg/cli/firewall_policy_test.go). Update this structure (and any examples below) so downstream gh run download users look in the right place.

Copilot · 2026-04-10T18:12:17Z

docs/src/content/docs/reference/compilation-process.md

 | **agent_usage.json** | `/tmp/gh-aw/` | Aggregated token counts: `{"input_tokens":…,"output_tokens":…,"cache_read_tokens":…,"cache_write_tokens":…}` | Bundled in the unified agent artifact when the firewall is enabled; accessible to third-party tools without parsing step summaries |
 | **prompt.txt** | `/tmp/gh-aw/aw-prompts/` | Generated prompt sent to AI agent (includes markdown instructions, imports, context variables) | Retained for debugging and reproduction |
+| **firewall-audit-logs** | See structure below | Dedicated artifact for AWF audit/observability logs (token usage, network policy, audit trail) | Uploaded by all firewall-enabled workflows; analyzed by `gh aw logs --artifacts firewall` |
 | **firewall-logs/** | `/tmp/gh-aw/firewall-logs/` | Network access logs in Squid format (when `network.firewall:` enabled) | Analyzed by `gh aw logs` command |


The table row for firewall-logs/ references /tmp/gh-aw/firewall-logs/, but the compiler uploads Squid logs from /tmp/gh-aw/sandbox/firewall/logs/ (see generateSquidLogsUploadStep in pkg/workflow/engine_firewall_support.go). Updating this path would keep the “Artifacts Created” table consistent with the actual runtime locations.

Suggested change

| **firewall-logs/** | `/tmp/gh-aw/firewall-logs/` | Network access logs in Squid format (when `network.firewall:` enabled) | Analyzed by `gh aw logs` command |

| **firewall-logs/** | `/tmp/gh-aw/sandbox/firewall/logs/` | Network access logs in Squid format (when `network.firewall:` enabled) | Analyzed by `gh aw logs` command |

Copilot · 2026-04-10T18:12:17Z

docs/src/content/docs/reference/artifacts.md

+  order: 298
+---
+
+GitHub Agentic Workflows uploads several artifacts during workflow execution. This reference documents every artifact name, its contents, and how to access the data — especially for downstream workflows that use `gh run download` directly instead of `gh aw logs`.


Grammar: “GitHub Agentic Workflows uploads” should be “GitHub Agentic Workflows upload”.

Suggested change

GitHub Agentic Workflows uploads several artifacts during workflow execution. This reference documents every artifact name, its contents, and how to access the data — especially for downstream workflows that use `gh run download` directly instead of `gh aw logs`.

GitHub Agentic Workflows upload several artifacts during workflow execution. This reference documents every artifact name, its contents, and how to access the data — especially for downstream workflows that use `gh run download` directly instead of `gh aw logs`.

Copilot · 2026-04-10T18:12:18Z

docs/src/content/docs/reference/compilation-process.md

+├── logs/
+│   ├── api-proxy-logs/
+│   │   └── token-usage.jsonl    ← Token usage data per request
+│   └── squid-logs/
+│       └── access.log           ← Network policy log (allow/deny)
+└── audit/
+    └── audit.jsonl              ← Firewall audit trail


The firewall-audit-logs structure uses logs/..., but the CLI expects token usage at firewall-audit-logs/api-proxy-logs/token-usage.jsonl (pkg/cli/token_usage.go + tests). Updating this tree will keep the compilation docs consistent with actual downloaded artifact layout.

This issue also appears on line 271 of the same file.

Suggested change

├── logs/

│ ├── api-proxy-logs/

│ │ └── token-usage.jsonl ← Token usage data per request

│ └── squid-logs/

│ └── access.log ← Network policy log (allow/deny)

└── audit/

└── audit.jsonl ← Firewall audit trail

├── api-proxy-logs/

│ └── token-usage.jsonl ← Token usage data per request

├── squid-logs/

│ └── access.log ← Network policy log (allow/deny)

└── audit.jsonl ← Firewall audit trail

Copilot · 2026-04-10T18:12:18Z

docs/src/content/docs/reference/artifacts.md

+| `safe-output` | `constants.SafeOutputArtifactName` | Single-file | Safe output data (`safe_output.jsonl`) |
+| `agent-output` | `constants.AgentOutputArtifactName` | Single-file | Agent output data (`agent_output.json`) |


The Quick Reference implies safe-output and agent-output are artifacts to download, but compiled workflows now merge those single-file artifacts into the unified agent artifact (see pkg/workflow/agentic_output_test.go assertions that the lock file must not contain name: safe-output). Consider marking these rows as legacy/back-compat so readers don’t try downloading them from new runs.

Suggested change

| `safe-output` | `constants.SafeOutputArtifactName` | Single-file | Safe output data (`safe_output.jsonl`) |

| `agent-output` | `constants.AgentOutputArtifactName` | Single-file | Agent output data (`agent_output.json`) |

| `safe-output` | `constants.SafeOutputArtifactName` | Legacy/back-compat | Historical standalone safe output artifact (`safe_output.jsonl`); in current compiled workflows this content is included in the unified `agent` artifact instead |

| `agent-output` | `constants.AgentOutputArtifactName` | Legacy/back-compat | Historical standalone agent output artifact (`agent_output.json`); in current compiled workflows this content is included in the unified `agent` artifact instead |

Copilot · 2026-04-10T18:12:18Z

docs/src/content/docs/reference/artifacts.md

+├── logs/
+│   ├── api-proxy-logs/
+│   │   └── token-usage.jsonl    ← Token usage data (input/output/cache tokens per API request)
+│   └── squid-logs/
+│       └── access.log           ← Network policy log (domain allow/deny decisions)


This firewall-audit-logs tree shows token usage under logs/api-proxy-logs/..., but the CLI/tests expect it directly under firewall-audit-logs/api-proxy-logs/token-usage.jsonl (pkg/cli/token_usage.go). Please update the documented structure to match the real extracted paths.

This issue also appears in the following locations of the same file:

line 86

line 107

Suggested change

├── logs/

│ ├── api-proxy-logs/

│ │ └── token-usage.jsonl ← Token usage data (input/output/cache tokens per API request)

│ └── squid-logs/

│ └── access.log ← Network policy log (domain allow/deny decisions)

├── api-proxy-logs/

│ └── token-usage.jsonl ← Token usage data (input/output/cache tokens per API request)

├── squid-logs/

│ └── access.log ← Network policy log (domain allow/deny decisions)

Copilot · 2026-04-10T18:12:19Z

docs/src/content/docs/reference/artifacts.md

+└── audit/
+    ├── audit.jsonl              ← Firewall audit trail (policy matches, rule evaluations)
+    └── policy-manifest.json     ← Policy configuration snapshot


The docs place firewall policy/audit files under audit/…, but the CLI’s firewall policy loader looks for policy-manifest.json and audit.jsonl at the root of the firewall-audit-logs directory (pkg/cli/firewall_policy.go/tests). Update these paths so copy/paste works for downstream consumers.

Suggested change

└── audit/

├── audit.jsonl ← Firewall audit trail (policy matches, rule evaluations)

└── policy-manifest.json ← Policy configuration snapshot

├── audit.jsonl ← Firewall audit trail (policy matches, rule evaluations)

└── policy-manifest.json ← Policy configuration snapshot

Copilot · 2026-04-10T18:12:19Z

docs/src/content/docs/reference/artifacts.md

+cat firewall-audit-logs/logs/squid-logs/access.log
+
+# Audit trail is at:
+cat firewall-audit-logs/audit/audit.jsonl


These examples use firewall-audit-logs/logs/... and firewall-audit-logs/audit/..., but the CLI expects firewall logs under firewall-audit-logs/squid-logs/access.log and policy/audit files at firewall-audit-logs/audit.jsonl + firewall-audit-logs/policy-manifest.json (pkg/cli/firewall_policy.go). Adjust these paths so downstream scripts locate the files correctly.

Suggested change

cat firewall-audit-logs/logs/squid-logs/access.log

# Audit trail is at:

cat firewall-audit-logs/audit/audit.jsonl

cat firewall-audit-logs/squid-logs/access.log

# Audit trail is at:

cat firewall-audit-logs/audit.jsonl

# Policy manifest is at:

cat firewall-audit-logs/policy-manifest.json

lpcox · 2026-04-10T18:12:45Z

@copilot apply changes based on the comments in this thread

- Fix directory structure to match CLI expectations: token usage at api-proxy-logs/ (not logs/api-proxy-logs/), audit.jsonl and policy-manifest.json at root (not audit/ subdirectory) - Move detection to single-file artifacts table (it only contains detection.log) - Clarify multi-file artifacts may be post-processed by CLI - Mark safe-output and agent-output as legacy/back-compat (now merged into unified agent artifact) - Fix grammar: "uploads" → "upload" - Fix firewall-logs path to /tmp/gh-aw/sandbox/firewall/logs/ Agent-Logs-Url: https://github.com/github/gh-aw/sessions/fbe8b2c5-5e0c-4a3d-8e42-a6bd64a1cbba Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot · 2026-04-10T18:20:42Z

@copilot apply changes based on the comments in this thread

Applied all changes from the review thread in ab25d02:

Fixed directory structure to match CLI expectations (api-proxy-logs/ at root, audit.jsonl and policy-manifest.json at root — no intermediate logs/ or audit/ directories)
Moved detection to single-file artifacts table
Clarified multi-file artifacts may be post-processed by CLI flattening helpers
Marked safe-output and agent-output as legacy/back-compat
Fixed grammar ("uploads" → "upload")
Fixed firewall-logs/ path to /tmp/gh-aw/sandbox/firewall/logs/

Initial plan

24e7d27

Copilot AI assigned Copilot and lpcox Apr 10, 2026

Copilot started work on behalf of lpcox April 10, 2026 17:56 View session

Copilot AI linked an issue Apr 10, 2026 that may be closed by this pull request

Doc: firewall-audit-logs artifact name not documented for downstream consumers #25683

Closed

Copilot AI changed the title ~~[WIP] Update documentation for firewall-audit-logs artifact name~~ Doc: document firewall-audit-logs artifact name for downstream consumers Apr 10, 2026

Copilot AI requested a review from lpcox April 10, 2026 18:05

Copilot finished work on behalf of lpcox April 10, 2026 18:05

lpcox marked this pull request as ready for review April 10, 2026 18:05

Copilot AI review requested due to automatic review settings April 10, 2026 18:05

Copilot started reviewing on behalf of lpcox April 10, 2026 18:06 View session

github-actions bot mentioned this pull request Apr 10, 2026

[aw] No-Op Runs #25214

Open

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Copilot started work on behalf of lpcox April 10, 2026 18:12 View session

Copilot finished work on behalf of lpcox April 10, 2026 18:21

lpcox merged commit cc56642 into main Apr 10, 2026

lpcox deleted the copilot/update-firewall-audit-logs-documentation branch April 10, 2026 18:57

	\| firewall-logs/ \| `/tmp/gh-aw/firewall-logs/` \| Network access logs in Squid format (when `network.firewall:` enabled) \| Analyzed by `gh aw logs` command \|
	\| firewall-logs/ \| `/tmp/gh-aw/sandbox/firewall/logs/` \| Network access logs in Squid format (when `network.firewall:` enabled) \| Analyzed by `gh aw logs` command \|

	GitHub Agentic Workflows uploads several artifacts during workflow execution. This reference documents every artifact name, its contents, and how to access the data — especially for downstream workflows that use `gh run download` directly instead of `gh aw logs`.
	GitHub Agentic Workflows upload several artifacts during workflow execution. This reference documents every artifact name, its contents, and how to access the data — especially for downstream workflows that use `gh run download` directly instead of `gh aw logs`.

		\| `safe-output` \| `constants.SafeOutputArtifactName` \| Single-file \| Safe output data (`safe_output.jsonl`) \|
		\| `agent-output` \| `constants.AgentOutputArtifactName` \| Single-file \| Agent output data (`agent_output.json`) \|

Conversation

Copilot AI commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Key documentation addition

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

lpcox commented Apr 10, 2026

Uh oh!

Copilot AI commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 10, 2026 •

edited

Loading