feat: compiler emits AWF JSON config file instead of CLI flag soup#29222
Open
feat: compiler emits AWF JSON config file instead of CLI flag soup#29222
Conversation
- Add `pkg/workflow/awf_config.go` with `BuildAWFConfigJSON()` function,
`awfSupportsConfigFile()` version gate, and `splitDomainList()` helper
- Add `AWFConfigFileMinVersion` constant to `pkg/constants/version_constants.go`
- Modify `BuildAWFCommand()` to write JSON config via printf and pass
`--config "${RUNNER_TEMP}/gh-aw/awf-config.json"` to AWF when supported
- Modify `BuildAWFArgs()` to omit flags now expressed in the config file
(--allow-domains, --block-domains, --enable-api-proxy, --image-tag,
--*-api-target) when the AWF version supports --config
- Full legacy fallback: old AWF versions keep all CLI flags unchanged
- Update golden files and all affected unit tests to reflect new behavior
- Add comprehensive tests in `pkg/workflow/awf_config_test.go`
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/7001ef78-eeee-4359-8de1-1cb092a2c990
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Migrate compiler to use AWF JSON config file instead of CLI flags
feat: compiler emits AWF JSON config file instead of CLI flag soup
Apr 30, 2026
Collaborator
|
@copilot merge main and recompile workflows |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the workflow compiler to generate and pass an AWF JSON config file via awf --config, replacing a large set of AWF CLI flags for network rules, API proxy settings, and container image tagging (with a version gate/fallback for older AWF versions).
Changes:
- Added AWF config JSON generation and wiring so compiled workflows write
${RUNNER_TEMP}/gh-aw/awf-config.jsonand invokeawf --config ...when supported. - Added
AWFConfigFileMinVersionconstant to semver-gate config-file usage and preserve legacy CLI-flag output for older pinned AWF versions. - Updated tests and regenerated golden/lock workflow outputs to reflect the new invocation pattern.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/awf_config.go | Introduces AWF config JSON structs + BuildAWFConfigJSON() and domain parsing helper. |
| pkg/constants/version_constants.go | Adds AWFConfigFileMinVersion for version-gating --config usage. |
| pkg/workflow/threat_detection_test.go | Updates API target assertions to look for the config JSON representation. |
| pkg/workflow/network_merge_import_test.go | Updates assertions to expect allowDomains in config JSON rather than CLI flags. |
| pkg/workflow/network_merge_edge_cases_test.go | Updates domain de-dup verification to inspect the JSON config content. |
| pkg/workflow/mcp_scripts_firewall_test.go | Updates firewall command structure checks to use config JSON. |
| pkg/workflow/gemini_engine_test.go | Updates expectations for allowDomains + apiProxy enabled to appear in config JSON. |
| pkg/workflow/firewall_workflow_test.go | Updates expectations for network configuration to appear in config JSON. |
| pkg/workflow/firewall_image_tag_test.go | Updates image tag assertions to expect imageTag in config JSON (and keeps a legacy fallback test). |
| pkg/workflow/firewall_blocked_domains_test.go | Updates blocked domain assertions to expect blockDomains in config JSON. |
| pkg/workflow/firewall_args_test.go | Updates firewall arg assertions to expect config JSON output. |
| pkg/workflow/enable_api_proxy_test.go | Updates API proxy assertions for config JSON; adds a legacy-version fallback test. |
| pkg/workflow/crush_engine_test.go | Updates firewall integration expectations to match config JSON behavior. |
| pkg/workflow/compiler_permissions_test.go | Updates permissions test expectations to match config JSON output. |
| pkg/workflow/claude_engine_network_test.go | Updates network-permission assertions to expect config JSON output. |
| pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden | Updates golden output for AWF invocation + config write. |
| pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden | Updates golden output for AWF invocation + config write. |
| .github/workflows/*.lock.yml (incl. regenerated workflow lock files) | Regenerates locked workflows to write AWF config JSON and call awf --config. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 105/225 changed files
- Comments generated: 2
Comment on lines
+28
to
+32
| // }, | ||
| // "container": { | ||
| // "imageTag": "v0.25.29,squid=sha256:..." | ||
| // } | ||
| // } |
There was a problem hiding this comment.
The schema example shows container.imageTag values with a leading "v" (e.g. "v0.25.29"), but getAWFImageTag() strips the v-prefix and the generated lock/golden outputs use "0.25.29". Update the example (or the ImageTag docs) to match the actual format to avoid confusing future readers.
Comment on lines
+176
to
+178
| // splitDomainList splits a comma-separated domain string into a sorted, deduplicated | ||
| // slice. Empty entries are ignored. The order of the original list is preserved for | ||
| // non-duplicate entries; this keeps the allow-list deterministic. |
There was a problem hiding this comment.
splitDomainList's docstring says it returns a "sorted, deduplicated" slice, but the implementation preserves the original order and does not sort. Either sort the result (and clarify the intended ordering) or adjust the comment to reflect the current behavior.
Contributor
There was a problem hiding this comment.
Good catch! The splitDomainList docstring claiming 'sorted, deduplicated' when the implementation only deduplicates (not sorts) is indeed misleading. Sorting would be a minor change but would help produce deterministic output across runs.
📰 BREAKING: Report filed by Smoke Copilot · ● 843.2K
…ler-to-awf-json-config # Conflicts: # .github/workflows/daily-cache-strategy-analyzer.lock.yml # .github/workflows/daily-doc-healer.lock.yml # .github/workflows/daily-doc-updater.lock.yml # .github/workflows/daily-fact.lock.yml # .github/workflows/daily-observability-report.lock.yml # .github/workflows/daily-rendering-scripts-verifier.lock.yml # .github/workflows/daily-safe-output-optimizer.lock.yml # .github/workflows/delight.lock.yml # .github/workflows/developer-docs-consolidator.lock.yml # .github/workflows/duplicate-code-detector.lock.yml # .github/workflows/go-fan.lock.yml # .github/workflows/go-logger.lock.yml # .github/workflows/grumpy-reviewer.lock.yml # .github/workflows/instructions-janitor.lock.yml # .github/workflows/issue-arborist.lock.yml # .github/workflows/safe-output-health.lock.yml # .github/workflows/schema-feature-coverage.lock.yml # .github/workflows/scout.lock.yml # .github/workflows/sergo.lock.yml # .github/workflows/smoke-call-workflow.lock.yml # .github/workflows/smoke-codex.lock.yml # .github/workflows/spec-enforcer.lock.yml # .github/workflows/step-name-alignment.lock.yml # .github/workflows/unbloat-docs.lock.yml Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Collaborator
|
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/4dee043f-af83-473e-81a8-d9061c574cce Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Contributor
Author
Done — merged main (989800c) and recompiled all 205 workflows successfully. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
…nfig.go Agent-Logs-Url: https://github.com/github/gh-aw/sessions/12c9457a-0152-4718-a73b-c93224066011 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Author
Main was already merged and workflows recompiled in the previous session. Addressed the two code review comments in 6426f64:
|
Collaborator
|
@copilot remove support for legacy flags, only config is supported |
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f4ba1722-039c-435a-a2db-e24fa1ba8b0c Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Author
Done in 1b854e4. Removed all legacy flag support:
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f4ba1722-039c-435a-a2db-e24fa1ba8b0c Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Collaborator
|
@copilot review all comments |
Contributor
Author
Contributor
|
|
Contributor
|
💫 TO BE CONTINUED... Smoke Claude failed! Our hero faces unexpected challenges... |
Contributor
|
📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing... |
Contributor
|
|
Contributor
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
Contributor
|
🚀 Smoke OpenCode MISSION COMPLETE! OpenCode delivered. 🔥 |
Contributor
Agent Container Tool Check
Result: 12/12 tools available ✅ Overall Status: PASS
|
Contributor
|
PRs: #29251 [jsweep] Clean remove_trigger_label.cjs; #29243 [docs] Unbloat: examples/project-tracking.md (-21%) Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "ab.chatgpt.com"
- "chatgpt.com"See Network Configuration for more information.
|
Contributor
|
Commit pushed:
|
Contributor
|
🎉 The smoke test agent was here! Just dropped by to say hello and verify all systems are go. Everything's running smoothly in the matrix! 🤖✨
|
Contributor
|
✅ Smoke Test #25148772853 results:
Overall: PASS 🎉
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
This PR is a solid refactor. The move from CLI flag lists to a JSON config file makes AWF invocations much cleaner, easier to audit in diffs, and sets the stage for schema validation in Phase 2. Left a couple of inline comments on permissions and documentation.
📰 BREAKING: Report filed by Smoke Copilot · ● 843.2K
| GH_AW_NODE_BIN=$(command -v node 2>/dev/null || true) | ||
| export GH_AW_NODE_BIN | ||
| (umask 177 && touch /tmp/gh-aw/agent-stdio.log) | ||
| printf '%s\n' '{"$schema":"https://github.com/github/gh-aw-firewall/schemas/awf-config.v1.json","network":{"allowDomains":["api.business.githubcopilot.com","api.enterprise.githubcopilot.com","api.github.com","api.githubcopilot.com","api.individual.githubcopilot.com","api.snapcraft.io","archive.ubuntu.com","azure.archive.ubuntu.com","crl.geotrust.com","crl.globalsign.com","crl.identrust.com","crl.sectigo.com","crl.thawte.com","crl.usertrust.com","crl.verisign.com","crl3.digicert.com","crl4.digicert.com","crls.ssl.com","github.com","host.docker.internal","json-schema.org","json.schemastore.org","keyserver.ubuntu.com","ocsp.digicert.com","ocsp.geotrust.com","ocsp.globalsign.com","ocsp.identrust.com","ocsp.sectigo.com","ocsp.ssl.com","ocsp.thawte.com","ocsp.usertrust.com","ocsp.verisign.com","packagecloud.io","packages.cloud.google.com","packages.microsoft.com","ppa.launchpad.net","raw.githubusercontent.com","registry.npmjs.org","s.symcb.com","s.symcd.com","security.ubuntu.com","telemetry.enterprise.githubcopilot.com","ts-crl.ws.symantec.com","ts-ocsp.ws.symantec.com","www.googleapis.com"]},"apiProxy":{"enabled":true},"container":{"imageTag":"0.25.29,squid=sha256:8a71ad9e40454051672312917e51567abfb8251d7c294d086c48f63d84e4cb53,agent=sha256:e68f37e36962dcb3f3d1de680a49bc2302cefd001b941a7dc377155ec7ce42f4,agent-act=sha256:97b4cc14dc2123a45b9d5b9927489f66882dec5857de6afc0e5bab257be92ef1,api-proxy=sha256:d1219e4110684402aabbeb5a43858f26790c9d0be210581cf3f7a521bd2c87b6,cli-proxy=sha256:29917488eb90a01ff9544ffeeb5cc26434a8ea16d69ae8972f5f6be0e567e276"}}' > "${RUNNER_TEMP}/gh-aw/awf-config.json" |
Contributor
There was a problem hiding this comment.
👍 This is a great improvement — writing the config to a JSON file via printf and passing --config to awf makes the AWF invocation much cleaner and auditable. One note: the JSON is written with umask 177 applied to agent-stdio.log but not to awf-config.json — consider using (umask 177 && printf ... > awf-config.json) to ensure the config file has restricted permissions, since it contains image digests and network allowlists.
| export GH_AW_NODE_BIN | ||
| (umask 177 && touch /tmp/gh-aw/agent-stdio.log) | ||
| printf '%s\n' '{"$schema":"https://github.com/github/gh-aw-firewall/schemas/awf-config.v1.json","network":{"allowDomains":["api.business.githubcopilot.com","api.enterprise.githubcopilot.com","api.github.com","api.githubcopilot.com","api.individual.githubcopilot.com","api.snapcraft.io","archive.ubuntu.com","azure.archive.ubuntu.com","crl.geotrust.com","crl.globalsign.com","crl.identrust.com","crl.sectigo.com","crl.thawte.com","crl.usertrust.com","crl.verisign.com","crl3.digicert.com","crl4.digicert.com","crls.ssl.com","github.com","host.docker.internal","json-schema.org","json.schemastore.org","keyserver.ubuntu.com","ocsp.digicert.com","ocsp.geotrust.com","ocsp.globalsign.com","ocsp.identrust.com","ocsp.sectigo.com","ocsp.ssl.com","ocsp.thawte.com","ocsp.usertrust.com","ocsp.verisign.com","packagecloud.io","packages.cloud.google.com","packages.microsoft.com","ppa.launchpad.net","raw.githubusercontent.com","registry.npmjs.org","s.symcb.com","s.symcd.com","security.ubuntu.com","telemetry.enterprise.githubcopilot.com","ts-crl.ws.symantec.com","ts-ocsp.ws.symantec.com","www.googleapis.com"]},"apiProxy":{"enabled":true},"container":{"imageTag":"0.25.29,squid=sha256:8a71ad9e40454051672312917e51567abfb8251d7c294d086c48f63d84e4cb53,agent=sha256:e68f37e36962dcb3f3d1de680a49bc2302cefd001b941a7dc377155ec7ce42f4,agent-act=sha256:97b4cc14dc2123a45b9d5b9927489f66882dec5857de6afc0e5bab257be92ef1,api-proxy=sha256:d1219e4110684402aabbeb5a43858f26790c9d0be210581cf3f7a521bd2c87b6,cli-proxy=sha256:29917488eb90a01ff9544ffeeb5cc26434a8ea16d69ae8972f5f6be0e567e276"}}' > "${RUNNER_TEMP}/gh-aw/awf-config.json" | ||
| # shellcheck disable=SC1003 |
Contributor
There was a problem hiding this comment.
The migration from flag-soup to --config is a nice cleanup. Since 205 lock files are being regenerated, it might be worth adding a note in CI or a README about running the lock file regeneration command, so contributors know how to update these files after changing workflow configurations.
Contributor
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
Collaborator
|
@copilot the final generated awf_config.json should be copied in the agent/detection artifact |
- Add AWFConfigFilePath constant (/tmp/gh-aw/awf-config.json)
- Copy awf-config.json from ${RUNNER_TEMP}/gh-aw/ to /tmp/gh-aw/ alongside
the printf write so it is included in the /tmp/gh-aw/ artifact root
- Add /tmp/gh-aw/awf-config.json to artifactPaths when firewall is enabled
- Update TestBuildAWFCommand_UsesConfigFile to assert the cp step is present
- Recompile all 205 workflows
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/5dc09588-98f3-41e4-bd90-7f78f916230e
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Contributor
Author
Done in 35f7122. The AWF config is written to Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AWF's
--config <path>support already exists but the compiler never used it, instead generating ever-longer flag lists that are fragile for complex values, hard to audit in diffs, and impossible to schema-validate at compile time.What changed
New:
pkg/workflow/awf_config.goBuildAWFConfigJSON()— serializesAWFCommandConfiginto compact AWF config JSON coveringnetwork(allowDomains, blockDomains),apiProxy(enabled, targets), andcontainer(imageTag)splitDomainList()— comma-separated domain parsing with deduplication (insertion order preserved)BuildAWFCommand()/BuildAWFArgs()inawf_helpers.go— the run step always writes the JSON viaprintfthen passes--configinstead of--allow-domains,--block-domains,--enable-api-proxy,--image-tag, and--*-api-targetflags. The config is also copied to/tmp/gh-aw/awf-config.jsonso it is included in the agent artifact.New:
pkg/constants/constants.go— addedAWFConfigFilePath = "/tmp/gh-aw/awf-config.json"constant for the artifact-side copy of the config file.205
.lock.ymlfiles regenerated. Before/after for the AWF invocation line:The agent artifact now includes
/tmp/gh-aw/awf-config.jsonalongside the proxy logs and audit directory, making the exact firewall configuration available for post-run analysis.Backwards compatibility
The legacy CLI flag path (
--allow-domains,--enable-api-proxy,--image-tag, API target flags) has been removed. All workflows unconditionally use--config.Scope
This implements Phase 1 of the issue (compiler generates config file). Phases 2 (schema validation at compile time) and 3 (composable config layers) are left for follow-up.
Changeset
--configinstead of expanded network, proxy, and container flags. The generated config file is also included in the unified agent artifact for post-run analysis.