Skip to content

fix: gh aw upgrade and gh aw compile produce identical lock files#19681

Merged
pelikhan merged 9 commits intomainfrom
copilot/fix-lock-file-toggle-issue
Mar 5, 2026
Merged

fix: gh aw upgrade and gh aw compile produce identical lock files#19681
pelikhan merged 9 commits intomainfrom
copilot/fix-lock-file-toggle-issue

Conversation

Copy link
Contributor

Copilot AI commented Mar 5, 2026

  • Fix root cause: compileAllWorkflowFiles uses relative paths while compileAllFilesInDirectory uses absolute paths
    • In pkg/cli/compile_helpers.go: resolve each file path to absolute with filepath.Abs before calling compileSingleFile
  • Fix Issue 1 (runtime-import path format): defensive fallback for paths starting with .github/ (no leading slash)
    • In pkg/workflow/compiler_yaml.go: handle paths starting with .github/ (no leading slash)
    • In pkg/parser/import_field_extractor.go: extract computeImportRelPath helper using strings.LastIndex to correctly handle repos named .github (e.g. /root/.github/.github/workflows/file.md)
  • Fix Issue 2 (dispatch_workflow empty inputs): automatic fix via absolute-path fix in compile_helpers.go
  • Fix Issue 3 (non-deterministic required array): sort required array in generateDispatchWorkflowTool
    • In pkg/workflow/safe_outputs_tools_generation.go: sort required array before assigning
  • Add test TestRuntimeImportPathRelative in compiler_yaml_test.go
  • Add test TestRuntimeImportPathGitHubIO in compiler_yaml_test.go covering various repo name structures
  • Add test TestGenerateDispatchWorkflowToolRequiredSorted in safe_outputs_tools_generation_test.go
  • Add TestComputeImportRelPath in pkg/parser/import_field_extractor_test.go with 17 table-driven cases covering variable file names, repo names (.github, .github.io, normal, github-in-name), subdirectory paths, relative paths, and fallback cases
  • Minimize duplication between compileAllWorkflowFiles and compileAllFilesInDirectory:
    • compileAllWorkflowFiles now uses getMarkdownWorkflowFiles instead of inline glob+filter
    • compileAllWorkflowFiles now uses saveActionCache+updateGitAttributes helpers instead of inline cache/gitattributes management
    • compileAllFilesInDirectory now uses getMarkdownWorkflowFiles instead of inline glob+filter
    • Fix regression: propagate errors from getMarkdownWorkflowFiles so TestCompileAllWorkflowFiles/compile_all_handles_glob_error passes
  • Revert actions/setup-cli/install.sh to main version (restore --pin and INSTALLED_VERSION output)
  • Merge main branch and recompile all 165 workflow lock files
Original prompt

This section details on the original issue you should resolve

<issue_title>Bug: gh aw upgrade and gh aw compile produce different lock files — toggle endlessly</issue_title>
<issue_description>## Summary

Running gh aw upgrade followed by gh aw compile (or vice versa) produces different output in .lock.yml files every time the commands are alternated. The files never converge to a stable state. This makes it impossible to commit a clean, idempotent upgrade.

Affected workflows:

  • .github/workflows/ld-flag-scanner.lock.yml
  • .github/workflows/ld-flag-cleanup-worker.lock.yml

Tool version: gh aw v0.53.2


Steps to Reproduce

# Starting from a clean state (HEAD has v0.52.1 lock files)
gh aw upgrade        # Produces state A (v0.53.2 lock files)
gh aw compile        # Produces state B (v0.53.2 lock files — but different)
gh aw upgrade        # Produces state A again
gh aw compile        # Produces state B again
# ...toggles indefinitely

Note: Both gh aw upgrade and gh aw compile report success with no errors. The issue is silent — there's no warning that the output is non-deterministic.


Observed Differences

Two distinct sections toggle on every command alternation in ld-flag-scanner.lock.yml. The same {{#runtime-import ...}} path toggling also occurs in ld-flag-cleanup-worker.lock.yml.

Diff 1 — {{#runtime-import}} path format

After gh aw upgrade:

          {{#runtime-import shared/ld-cleanup-shared-tools.md}}
          ...
          {{#runtime-import ld-flag-scanner.md}}

After gh aw compile:

          {{#runtime-import .github/workflows/shared/ld-cleanup-shared-tools.md}}
          ...
          {{#runtime-import .github/workflows/ld-flag-scanner.md}}

gh aw upgrade emits short relative import paths (relative to .github/workflows/), while gh aw compile emits full repo-root-anchored paths (prefixed with .github/workflows/). Both are valid representations, but the two commands disagree on which form to use.

This affects every {{#runtime-import ...}} directive in every workflow. In ld-flag-cleanup-worker.lock.yml, three imports toggle (including shared/pnpm.md).


Diff 2 — dispatch_workflow input schema for ld_flag_cleanup_worker

After gh aw upgrade:

{
  "name": "ld_flag_cleanup_worker",
  "description": "Dispatch the 'ld-flag-cleanup-worker' workflow ...",
  "inputSchema": {
    "additionalProperties": false,
    "properties": {},
    "type": "object"
  }
}

After gh aw compile:

{
  "name": "ld_flag_cleanup_worker",
  "description": "Dispatch the 'ld-flag-cleanup-worker' workflow ...",
  "inputSchema": {
    "additionalProperties": false,
    "properties": {
      "flag_key": {
        "description": "The LaunchDarkly flag key to clean up",
        "type": "string"
      },
      "tracker_issue": {
        "description": "Dashboard issue number to reference",
        "type": "string"
      }
    },
    "required": [
      "flag_key",
      "tracker_issue"
    ],
    "type": "object"
  }
}

The ld-flag-cleanup-worker.md source file declares workflow_dispatch.inputs with flag_key and tracker_issue. When compiled via standalone gh aw compile, these inputs are reflected in the dispatch_workflow tool's inputSchema (correct behaviour). When compiled via gh aw upgrade, the inputs are stripped and "properties": {} is emitted (broken — the agent cannot know what inputs to pass when dispatching the workflow).

Additional observation: Even when the inputs are present (post-gh aw compile), the order of elements in the "required" array is not stable across successive gh aw compile runs. The array has been observed to flip between ["flag_key", "tracker_issue"] and ["tracker_issue", "flag_key"]. This suggests the input schema is being derived from an unordered data structure (e.g. an object/map) whose key iteration order is non-deterministic, rather than from the declared order in the source .md file.


Analysis / Probable Causes

Issue 1 — Runtime import path format

The compiler appears to use a different working/base directory when invoked by gh aw upgrade vs standalone gh aw compile. If the compiler resolves imports relative to .github/workflows/, it emits the short form. If it resolves from the repo root, it emits the .github/workflows/-prefixed form. This is likely a difference in the cwd passed to the compiler internals between the two code paths.

Issue 2 — dispatch_workflow input schema

gh aw upgrade calls gh aw compile internally for each workflow in sequence. Its output order confirms: ld-flag-cleanup-worker.md is compiled before ld-flag-scanner.md. The scanner's safe_outputs config derives the dispatch_workflow input schema by inspecting the dispatched workflow's declared workflow_dispatch.inputs.

The discrepancy suggests that when called via upgrade, the comp...


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits March 5, 2026 03:57
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix locking file toggle issue in gh aw commands fix: gh aw upgrade and gh aw compile produce identical lock files Mar 5, 2026
// Imports with inputs must be inlined for compile-time substitution.
// Extract relative path from repository root (from .github/ onwards).
var importRelPath string
if idx := strings.Index(item.fullPath, "/.github/"); idx >= 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot make sure to handle case where the repo name is ".github"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 30d1f16. Changed strings.Index to strings.LastIndex when searching for /.github/ in import_field_extractor.go, matching the same approach already used in compiler_yaml.go. For a repo named .github with a path like /root/.github/.github/workflows/file.md, LastIndex correctly finds the actual workflows directory rather than the repo root.

@pelikhan pelikhan marked this pull request as ready for review March 5, 2026 04:18
Copilot AI review requested due to automatic review settings March 5, 2026 04:18
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes non-deterministic .lock.yml output differences between gh aw upgrade and gh aw compile by normalizing workflow file paths and stabilizing generated schemas.

Changes:

  • Resolve workflow .md files to absolute paths before compilation to keep runtime-import paths and dispatch input extraction consistent.
  • Add defensive handling for already-.github/... relative paths in prompt/runtime-import and import-field extraction.
  • Sort dispatch_workflow tool-schema required fields for deterministic output; add regression tests for both fixes.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/cli/compile_helpers.go Normalize workflow file paths to absolute before compiling in bulk.
pkg/workflow/compiler_yaml.go Preserve .github/... relative markdown paths when generating runtime-import macros.
pkg/parser/import_field_extractor.go Preserve .github/... relative paths when computing import-relative paths.
pkg/workflow/safe_outputs_tools_generation.go Sort tool-schema required fields for stable JSON output.
pkg/workflow/safe_outputs_tools_generation_test.go Adds regression test ensuring sorted required output.
pkg/workflow/compiler_yaml_test.go Adds regression test ensuring relative-path compilation emits .github/... runtime-import paths.
actions/setup-cli/install.sh Modifies gh-extension install path and GitHub Actions outputs (see comments).
.github/workflows/constraint-solving-potd.lock.yml Regenerated lockfile changes (agent/mcpg versions + added job concurrency).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +242 to +243
# Try to install using gh
if gh extension install "$REPO" --force 2>&1 | tee /tmp/gh-install.log; then
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The if gh extension install ... | tee ...; then condition is checking the exit status of tee, not gh extension install (since the script doesn’t set pipefail). This can report success even when the install fails. Consider enabling set -o pipefail for the script or restoring the prior PIPESTATUS[0] handling so failures are detected correctly (and preserve --pin "$VERSION" behavior when a specific version is requested).

Suggested change
# Try to install using gh
if gh extension install "$REPO" --force 2>&1 | tee /tmp/gh-install.log; then
# Build gh extension install command (pin version if specified)
GH_INSTALL_CMD=(gh extension install "$REPO" --force)
if [ -n "$VERSION" ] && [ "$VERSION" != "latest" ]; then
GH_INSTALL_CMD+=(--pin "$VERSION")
fi
# Try to install using gh and capture the exit status of gh (not tee)
"${GH_INSTALL_CMD[@]}" 2>&1 | tee /tmp/gh-install.log
gh_install_status=${PIPESTATUS[0]}
if [ "$gh_install_status" -eq 0 ]; then

Copilot uses AI. Check for mistakes.
# Set output for GitHub Actions
if [ -n "${GITHUB_OUTPUT}" ]; then
echo "installed_version=${INSTALLED_VERSION}" >> "${GITHUB_OUTPUT}"
echo "installed_version=${VERSION}" >> "${GITHUB_OUTPUT}"
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

installed_version is currently written as ${VERSION}, but the script already computes ${INSTALLED_VERSION} from gh aw version. If VERSION is latest (or differs from what gh actually installed), this output will be wrong/empty. Write ${INSTALLED_VERSION} to GITHUB_OUTPUT here to reflect the real installed version.

Suggested change
echo "installed_version=${VERSION}" >> "${GITHUB_OUTPUT}"
echo "installed_version=${INSTALLED_VERSION}" >> "${GITHUB_OUTPUT}"

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@pelikhan
Copy link
Contributor

pelikhan commented Mar 5, 2026

@copilot revert install.sh and merge main, recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot revert install.sh and merge main, recompile

Done in d7cc7de. Reverted actions/setup-cli/install.sh (and its source install-gh-aw.sh) to main's version (restoring --pin "$VERSION" support, PIPESTATUS exit-status handling, and installed_version=${INSTALLED_VERSION} output), merged main, and recompiled all 165 workflow lock files.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
  • https://api.github.com/repos/actions/download-artifact/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/download-artifact/git/ref/tags/v8 --jq .object.sha (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha ithub/workflows --local cal/bin/bash credential.usern/opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/a70c5eada06553e3510ac27f2c3bda9d3705bccb
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/a70c5eada06553e3510ac27f2c3bda9d3705bccb --jq .object.sha (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan
Copy link
Contributor

pelikhan commented Mar 5, 2026

@copilot add tests with variable file name / repo name structure

… name structures

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot add tests with variable file name / repo name structure

Done in cf64029. Extracted the importRelPath computation into a computeImportRelPath helper in import_field_extractor.go and added TestComputeImportRelPath in pkg/parser/import_field_extractor_test.go with 17 table-driven cases covering:

  • Normal absolute paths (root-level and subdirectory files)
  • Repo named .github (verifies LastIndex picks the right segment)
  • GitHub Pages repos (.github.io suffix)
  • Repos with "github" anywhere in the name
  • Relative paths already starting with .github/
  • Special file names (hyphens, underscores, dots)
  • Files in shared subdirectories
  • Windows-style backslash paths (documents Linux fallback behaviour)
  • Paths outside .github/ (fallback to importPath)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE ortcfg env g/envutil/envutiGOINSECURE g/envutil/envutiGOMOD 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan
Copy link
Contributor

pelikhan commented Mar 5, 2026

@copilot try to minimize the overlap/duplication of code between compileAllWorkflowFiles and compileAllFilesInDirectory

…ompileAllFilesInDirectory

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot try to minimize the overlap/duplication of code between compileAllWorkflowFiles and compileAllFilesInDirectory

Done in ddfe4f4. Reduced ~53 lines of duplicated logic across the two functions:

  • compileAllWorkflowFiles (compile_helpers.go): replaced inline filepath.Glob+filterWorkflowFiles with getMarkdownWorkflowFiles, and replaced the ~20-line inline action-cache-save + gitattributes-update block with calls to the already-existing saveActionCache and updateGitAttributes helpers.
  • compileAllFilesInDirectory (compile_orchestration.go): replaced inline filepath.Glob+filterWorkflowFiles with getMarkdownWorkflowFiles.

Both functions now share the same file-scanning and post-compile cleanup code paths.

@pelikhan
Copy link
Contributor

pelikhan commented Mar 5, 2026

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot fix tests https://github.com/github/gh-aw/actions/runs/22702945204/job/65824091321

Fixed in 3ec351b. The refactoring in the previous commit was silently swallowing errors from getMarkdownWorkflowFiles (returning nil in both the error and empty-files cases), which caused TestCompileAllWorkflowFiles/compile_all_handles_glob_error to panic when it tried to call .Error() on the nil error.

Split the two cases: now errors from getMarkdownWorkflowFiles are propagated as "failed to find markdown files: ...", while the empty-files case still returns stats, nil.

@pelikhan pelikhan merged commit 6dc2fa2 into main Mar 5, 2026
82 checks passed
@pelikhan pelikhan deleted the copilot/fix-lock-file-toggle-issue branch March 5, 2026 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: gh aw upgrade and gh aw compile produce different lock files — toggle endlessly

3 participants