fix: gh aw upgrade and gh aw compile produce identical lock files#19681
fix: gh aw upgrade and gh aw compile produce identical lock files#19681
gh aw upgrade and gh aw compile produce identical lock files#19681Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
gh aw upgrade and gh aw compile produce identical lock files
| // Imports with inputs must be inlined for compile-time substitution. | ||
| // Extract relative path from repository root (from .github/ onwards). | ||
| var importRelPath string | ||
| if idx := strings.Index(item.fullPath, "/.github/"); idx >= 0 { |
There was a problem hiding this comment.
@copilot make sure to handle case where the repo name is ".github"
There was a problem hiding this comment.
Fixed in 30d1f16. Changed strings.Index to strings.LastIndex when searching for /.github/ in import_field_extractor.go, matching the same approach already used in compiler_yaml.go. For a repo named .github with a path like /root/.github/.github/workflows/file.md, LastIndex correctly finds the actual workflows directory rather than the repo root.
There was a problem hiding this comment.
Pull request overview
Fixes non-deterministic .lock.yml output differences between gh aw upgrade and gh aw compile by normalizing workflow file paths and stabilizing generated schemas.
Changes:
- Resolve workflow
.mdfiles to absolute paths before compilation to keep runtime-import paths and dispatch input extraction consistent. - Add defensive handling for already-
.github/...relative paths in prompt/runtime-import and import-field extraction. - Sort
dispatch_workflowtool-schemarequiredfields for deterministic output; add regression tests for both fixes.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
pkg/cli/compile_helpers.go |
Normalize workflow file paths to absolute before compiling in bulk. |
pkg/workflow/compiler_yaml.go |
Preserve .github/... relative markdown paths when generating runtime-import macros. |
pkg/parser/import_field_extractor.go |
Preserve .github/... relative paths when computing import-relative paths. |
pkg/workflow/safe_outputs_tools_generation.go |
Sort tool-schema required fields for stable JSON output. |
pkg/workflow/safe_outputs_tools_generation_test.go |
Adds regression test ensuring sorted required output. |
pkg/workflow/compiler_yaml_test.go |
Adds regression test ensuring relative-path compilation emits .github/... runtime-import paths. |
actions/setup-cli/install.sh |
Modifies gh-extension install path and GitHub Actions outputs (see comments). |
.github/workflows/constraint-solving-potd.lock.yml |
Regenerated lockfile changes (agent/mcpg versions + added job concurrency). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
actions/setup-cli/install.sh
Outdated
| # Try to install using gh | ||
| if gh extension install "$REPO" --force 2>&1 | tee /tmp/gh-install.log; then |
There was a problem hiding this comment.
The if gh extension install ... | tee ...; then condition is checking the exit status of tee, not gh extension install (since the script doesn’t set pipefail). This can report success even when the install fails. Consider enabling set -o pipefail for the script or restoring the prior PIPESTATUS[0] handling so failures are detected correctly (and preserve --pin "$VERSION" behavior when a specific version is requested).
| # Try to install using gh | |
| if gh extension install "$REPO" --force 2>&1 | tee /tmp/gh-install.log; then | |
| # Build gh extension install command (pin version if specified) | |
| GH_INSTALL_CMD=(gh extension install "$REPO" --force) | |
| if [ -n "$VERSION" ] && [ "$VERSION" != "latest" ]; then | |
| GH_INSTALL_CMD+=(--pin "$VERSION") | |
| fi | |
| # Try to install using gh and capture the exit status of gh (not tee) | |
| "${GH_INSTALL_CMD[@]}" 2>&1 | tee /tmp/gh-install.log | |
| gh_install_status=${PIPESTATUS[0]} | |
| if [ "$gh_install_status" -eq 0 ]; then |
actions/setup-cli/install.sh
Outdated
| # Set output for GitHub Actions | ||
| if [ -n "${GITHUB_OUTPUT}" ]; then | ||
| echo "installed_version=${INSTALLED_VERSION}" >> "${GITHUB_OUTPUT}" | ||
| echo "installed_version=${VERSION}" >> "${GITHUB_OUTPUT}" |
There was a problem hiding this comment.
installed_version is currently written as ${VERSION}, but the script already computes ${INSTALLED_VERSION} from gh aw version. If VERSION is latest (or differs from what gh actually installed), this output will be wrong/empty. Write ${INSTALLED_VERSION} to GITHUB_OUTPUT here to reflect the real installed version.
| echo "installed_version=${VERSION}" >> "${GITHUB_OUTPUT}" | |
| echo "installed_version=${INSTALLED_VERSION}" >> "${GITHUB_OUTPUT}" |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@copilot revert install.sh and merge main, recompile |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details. Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in d7cc7de. Reverted Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
|
@copilot add tests with variable file name / repo name structure |
… name structures Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in cf64029. Extracted the
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
|
@copilot try to minimize the overlap/duplication of code between compileAllWorkflowFiles and compileAllFilesInDirectory |
…ompileAllFilesInDirectory Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in ddfe4f4. Reduced ~53 lines of duplicated logic across the two functions:
Both functions now share the same file-scanning and post-compile cleanup code paths. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in 3ec351b. The refactoring in the previous commit was silently swallowing errors from Split the two cases: now errors from |
compileAllWorkflowFilesuses relative paths whilecompileAllFilesInDirectoryuses absolute pathspkg/cli/compile_helpers.go: resolve each file path to absolute withfilepath.Absbefore callingcompileSingleFile.github/(no leading slash)pkg/workflow/compiler_yaml.go: handle paths starting with.github/(no leading slash)pkg/parser/import_field_extractor.go: extractcomputeImportRelPathhelper usingstrings.LastIndexto correctly handle repos named.github(e.g./root/.github/.github/workflows/file.md)dispatch_workflowempty inputs): automatic fix via absolute-path fix in compile_helpers.gorequiredarray): sortrequiredarray ingenerateDispatchWorkflowToolpkg/workflow/safe_outputs_tools_generation.go: sortrequiredarray before assigningTestRuntimeImportPathRelativeincompiler_yaml_test.goTestRuntimeImportPathGitHubIOincompiler_yaml_test.gocovering various repo name structuresTestGenerateDispatchWorkflowToolRequiredSortedinsafe_outputs_tools_generation_test.goTestComputeImportRelPathinpkg/parser/import_field_extractor_test.gowith 17 table-driven cases covering variable file names, repo names (.github,.github.io, normal,github-in-name), subdirectory paths, relative paths, and fallback casescompileAllWorkflowFilesandcompileAllFilesInDirectory:compileAllWorkflowFilesnow usesgetMarkdownWorkflowFilesinstead of inline glob+filtercompileAllWorkflowFilesnow usessaveActionCache+updateGitAttributeshelpers instead of inline cache/gitattributes managementcompileAllFilesInDirectorynow usesgetMarkdownWorkflowFilesinstead of inline glob+filtergetMarkdownWorkflowFilessoTestCompileAllWorkflowFiles/compile_all_handles_glob_errorpassesactions/setup-cli/install.shto main version (restore--pinandINSTALLED_VERSIONoutput)Original prompt
This section details on the original issue you should resolve
<issue_title>Bug:
gh aw upgradeandgh aw compileproduce different lock files — toggle endlessly</issue_title><issue_description>## Summary
Running
gh aw upgradefollowed bygh aw compile(or vice versa) produces different output in.lock.ymlfiles every time the commands are alternated. The files never converge to a stable state. This makes it impossible to commit a clean, idempotent upgrade.Affected workflows:
.github/workflows/ld-flag-scanner.lock.yml.github/workflows/ld-flag-cleanup-worker.lock.ymlTool version:
gh awv0.53.2Steps to Reproduce
Observed Differences
Two distinct sections toggle on every command alternation in
ld-flag-scanner.lock.yml. The same{{#runtime-import ...}}path toggling also occurs inld-flag-cleanup-worker.lock.yml.Diff 1 —
{{#runtime-import}}path formatAfter
gh aw upgrade:After
gh aw compile:gh aw upgradeemits short relative import paths (relative to.github/workflows/), whilegh aw compileemits full repo-root-anchored paths (prefixed with.github/workflows/). Both are valid representations, but the two commands disagree on which form to use.This affects every
{{#runtime-import ...}}directive in every workflow. Inld-flag-cleanup-worker.lock.yml, three imports toggle (includingshared/pnpm.md).Diff 2 —
dispatch_workflowinput schema forld_flag_cleanup_workerAfter
gh aw upgrade:{ "name": "ld_flag_cleanup_worker", "description": "Dispatch the 'ld-flag-cleanup-worker' workflow ...", "inputSchema": { "additionalProperties": false, "properties": {}, "type": "object" } }After
gh aw compile:{ "name": "ld_flag_cleanup_worker", "description": "Dispatch the 'ld-flag-cleanup-worker' workflow ...", "inputSchema": { "additionalProperties": false, "properties": { "flag_key": { "description": "The LaunchDarkly flag key to clean up", "type": "string" }, "tracker_issue": { "description": "Dashboard issue number to reference", "type": "string" } }, "required": [ "flag_key", "tracker_issue" ], "type": "object" } }The
ld-flag-cleanup-worker.mdsource file declaresworkflow_dispatch.inputswithflag_keyandtracker_issue. When compiled via standalonegh aw compile, these inputs are reflected in thedispatch_workflowtool'sinputSchema(correct behaviour). When compiled viagh aw upgrade, the inputs are stripped and"properties": {}is emitted (broken — the agent cannot know what inputs to pass when dispatching the workflow).Analysis / Probable Causes
Issue 1 — Runtime import path format
The compiler appears to use a different working/base directory when invoked by
gh aw upgradevs standalonegh aw compile. If the compiler resolves imports relative to.github/workflows/, it emits the short form. If it resolves from the repo root, it emits the.github/workflows/-prefixed form. This is likely a difference in thecwdpassed to the compiler internals between the two code paths.Issue 2 —
dispatch_workflowinput schemagh aw upgradecallsgh aw compileinternally for each workflow in sequence. Its output order confirms:ld-flag-cleanup-worker.mdis compiled beforeld-flag-scanner.md. The scanner'ssafe_outputsconfig derives thedispatch_workflowinput schema by inspecting the dispatched workflow's declaredworkflow_dispatch.inputs.The discrepancy suggests that when called via upgrade, the comp...
gh aw upgradeandgh aw compileproduce different lock files — toggle endlessly #19622💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.