Skip to content

[q] fix: install crush to writable npm prefix to fix EROFS in AWF chroot mode#29734

Closed
github-actions[bot] wants to merge 1 commit intomainfrom
q/fix-crush-erofs-writable-prefix-11ef079cc8bc2546
Closed

[q] fix: install crush to writable npm prefix to fix EROFS in AWF chroot mode#29734
github-actions[bot] wants to merge 1 commit intomainfrom
q/fix-crush-erofs-writable-prefix-11ef079cc8bc2546

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot commented May 2, 2026

Q Workflow Optimization Report

Issues Found (from live data)

Smoke Crush

  • Run ID Analyzed: 25248852737
  • Issue: Error: Installation failed: EROFS: read-only file system, mkdir '/opt/hostedtoolcache/node/24.14.1/x64/lib/node_modules/@charmland/crush/bin'
  • Root Cause: The @charmland/crush npm package lazily downloads the actual binary at runtime when crush run is first invoked. It tries to write the binary into its own package directory, which lives inside the npm global prefix (/opt/hostedtoolcache/node/.../lib/node_modules/). In AWF chroot mode, the host filesystem is mounted read-only (EROFS), causing the download to fail.

Changes Made

pkg/workflow/crush_engine.go

  1. Install to writable prefix: Changed the crush installation step from:

    npm install --ignore-scripts -g `@charmland/crush`@X.Y.Z
    

    to:

    NPM_CONFIG_PREFIX=/tmp/npm-global npm install --ignore-scripts -g `@charmland/crush`@X.Y.Z
    

    This installs the crush package to /tmp/npm-global/lib/node_modules/, which is always writable — both in standard execution and in AWF chroot mode.

  2. PATH update: Prepend /tmp/npm-global/bin to the PATH in both the firewall-enabled and non-firewall execution paths so the crush binary can be found at runtime.

Other Findings (No Action Required)

  • Terminal Stylist (run 25248339422): 16 "blocked" requests were actually error:transaction-end-before-headers from ::1 (localhost health checks), not real outbound blocks. No workflow config change needed.
  • Design Decision Gate: Variable turn count (0–5) is expected behavior for PRs that may or may not need ADR review.
  • No missing tools across all 20 analyzed runs.
  • No permission errors in any run.

Expected Improvements

  • Smoke Crush will no longer fail with EROFS errors in AWF chroot mode
  • The fix is backward-compatible — non-chroot environments also benefit since /tmp/npm-global is writable everywhere

Note on Lock Files

Lock files (.lock.yml) will be regenerated automatically by CI after this PR is merged, since the Go binary needs to be rebuilt first.

References

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • charm.land
  • proxy.golang.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "charm.land"
    - "proxy.golang.org"

See Network Configuration for more information.

🎩 Equipped by Q · ● 9.6M ·

  • expires on May 4, 2026, 9:47 AM UTC

…mode

The @charmland/crush npm package lazily downloads the actual binary into
its package directory at runtime (when `crush run` is first invoked).
In AWF chroot mode, the standard npm global prefix
(/opt/hostedtoolcache/node/.../lib/node_modules/) is mounted read-only,
causing the binary download to fail with:

  Error: Installation failed: EROFS: read-only file system,
  mkdir '/opt/hostedtoolcache/node/.../lib/node_modules/@charmland/crush/bin'

Fix by setting NPM_CONFIG_PREFIX=/tmp/npm-global in the install step so
the crush package and its binary land in a writable /tmp directory.
Also prepend /tmp/npm-global/bin to PATH in the execution steps so the
installed crush binary is found both in AWF chroot mode and in
non-firewall execution.

Evidence: run #25248852737 (Smoke Crush), identified by Q weekly analysis.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant