Skip to content

Bug: gh aw upgrade and gh aw compile produce different lock files — toggle endlessly #19622

@srgibbs99

Description

@srgibbs99

Summary

Running gh aw upgrade followed by gh aw compile (or vice versa) produces different output in .lock.yml files every time the commands are alternated. The files never converge to a stable state. This makes it impossible to commit a clean, idempotent upgrade.

Affected workflows:

  • .github/workflows/ld-flag-scanner.lock.yml
  • .github/workflows/ld-flag-cleanup-worker.lock.yml

Tool version: gh aw v0.53.2


Steps to Reproduce

# Starting from a clean state (HEAD has v0.52.1 lock files)
gh aw upgrade        # Produces state A (v0.53.2 lock files)
gh aw compile        # Produces state B (v0.53.2 lock files — but different)
gh aw upgrade        # Produces state A again
gh aw compile        # Produces state B again
# ...toggles indefinitely

Note: Both gh aw upgrade and gh aw compile report success with no errors. The issue is silent — there's no warning that the output is non-deterministic.


Observed Differences

Two distinct sections toggle on every command alternation in ld-flag-scanner.lock.yml. The same {{#runtime-import ...}} path toggling also occurs in ld-flag-cleanup-worker.lock.yml.

Diff 1 — {{#runtime-import}} path format

After gh aw upgrade:

          {{#runtime-import shared/ld-cleanup-shared-tools.md}}
          ...
          {{#runtime-import ld-flag-scanner.md}}

After gh aw compile:

          {{#runtime-import .github/workflows/shared/ld-cleanup-shared-tools.md}}
          ...
          {{#runtime-import .github/workflows/ld-flag-scanner.md}}

gh aw upgrade emits short relative import paths (relative to .github/workflows/), while gh aw compile emits full repo-root-anchored paths (prefixed with .github/workflows/). Both are valid representations, but the two commands disagree on which form to use.

This affects every {{#runtime-import ...}} directive in every workflow. In ld-flag-cleanup-worker.lock.yml, three imports toggle (including shared/pnpm.md).


Diff 2 — dispatch_workflow input schema for ld_flag_cleanup_worker

After gh aw upgrade:

{
  "name": "ld_flag_cleanup_worker",
  "description": "Dispatch the 'ld-flag-cleanup-worker' workflow ...",
  "inputSchema": {
    "additionalProperties": false,
    "properties": {},
    "type": "object"
  }
}

After gh aw compile:

{
  "name": "ld_flag_cleanup_worker",
  "description": "Dispatch the 'ld-flag-cleanup-worker' workflow ...",
  "inputSchema": {
    "additionalProperties": false,
    "properties": {
      "flag_key": {
        "description": "The LaunchDarkly flag key to clean up",
        "type": "string"
      },
      "tracker_issue": {
        "description": "Dashboard issue number to reference",
        "type": "string"
      }
    },
    "required": [
      "flag_key",
      "tracker_issue"
    ],
    "type": "object"
  }
}

The ld-flag-cleanup-worker.md source file declares workflow_dispatch.inputs with flag_key and tracker_issue. When compiled via standalone gh aw compile, these inputs are reflected in the dispatch_workflow tool's inputSchema (correct behaviour). When compiled via gh aw upgrade, the inputs are stripped and "properties": {} is emitted (broken — the agent cannot know what inputs to pass when dispatching the workflow).

Additional observation: Even when the inputs are present (post-gh aw compile), the order of elements in the "required" array is not stable across successive gh aw compile runs. The array has been observed to flip between ["flag_key", "tracker_issue"] and ["tracker_issue", "flag_key"]. This suggests the input schema is being derived from an unordered data structure (e.g. an object/map) whose key iteration order is non-deterministic, rather than from the declared order in the source .md file.


Analysis / Probable Causes

Issue 1 — Runtime import path format

The compiler appears to use a different working/base directory when invoked by gh aw upgrade vs standalone gh aw compile. If the compiler resolves imports relative to .github/workflows/, it emits the short form. If it resolves from the repo root, it emits the .github/workflows/-prefixed form. This is likely a difference in the cwd passed to the compiler internals between the two code paths.

Issue 2 — dispatch_workflow input schema

gh aw upgrade calls gh aw compile internally for each workflow in sequence. Its output order confirms: ld-flag-cleanup-worker.md is compiled before ld-flag-scanner.md. The scanner's safe_outputs config derives the dispatch_workflow input schema by inspecting the dispatched workflow's declared workflow_dispatch.inputs.

The discrepancy suggests that when called via upgrade, the compiler reads the inputs from the already-compiled lock file of the worker (which at that point in the upgrade pass may still be the old v0.52.1 lock file or an intermediate state), rather than the source .md file. When called via standalone gh aw compile, it reads from the source .md — and correctly includes flag_key and tracker_issue.

Alternatively, the upgrade path may intentionally strip dispatch_workflow inputs as part of a migration codemod, while compile preserves them from the source — either way the two commands are not idempotent with each other.


Impact

  • Non-idempotent lock files: PRs that run gh aw upgrade in CI will always produce a diff against whatever was already committed, making it impossible to detect "already up to date" state.
  • Functional regression: When gh aw upgrade is the last command run, the ld_flag_cleanup_worker dispatch tool has empty input properties — the agent has no schema to follow when constructing the dispatch call, so it cannot correctly pass flag_key and tracker_issue at runtime.
  • CI noise: Any automation that runs both commands (upgrade-then-compile or commit-then-validate) will see perpetual dirty state.

Expected Behaviour

gh aw upgrade followed by gh aw compile (or any combination thereof) should produce identical, stable output. A compiled lock file should be idempotent: running gh aw compile on an already-compiled file should produce no diff.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions