fix(workflows): restore Actions Expression interpolation in Claude prompts#5520
Merged
MarkusNeusinger merged 1 commit intomainfrom Apr 29, 2026
Merged
Conversation
…ompts Three workflows passed runtime values to claude-code-action via shell-style `$VAR` placeholders inside the `with: prompt: |` block. That block is a YAML input to a JavaScript Action — there is no shell, so `$VAR` is never expanded. Claude received literal `$LIBRARY` / `$ISSUE_TITLE` etc. instead of the actual values, and silently failed (no `quality_score.txt` produced; review step exited "success" but validate step then exit 1 with `ai-review-failed`). Symptom seen on `main`: - 4 of 5 stuck PRs (#5476, #5483, #5486, ~#5480 sometimes) hit `ai-review-failed` repeatedly. Run logs show review step running ~20s with literal `$LIBRARY` in the PROMPT echo. 5480 occasionally recovered because Claude inferred values from cwd, hence the intermittent pattern. - spec-create / report-validate not yet observed failing only because no triggering issue has come in since 2026-04-28; both would fail the same way. Root cause: 252977c ("chore: fix critical audit findings") attempted to mitigate workflow injection by routing `${{ github.event.* }}` through `env:` and dereferencing as `$VAR`. That is the correct mitigation for shell `run:` steps and Python heredocs (and those changes are kept). Inside `with: prompt: |` it is wrong: the value is a YAML string handed to a Node/Bun action, not executed by a shell, so there is no injection surface and `$VAR` does not interpolate. Fix: revert *only* the descriptive header lines of each `prompt:` block to use `${{ ... }}` Actions Expressions, which the runner substitutes into the prompt string before the action receives it. The shell-style `$VAR` references inside embedded bash code samples (e.g. `gh issue edit $ISSUE_NUMBER`) are left as-is — they are run by Claude's Bash tool which inherits the step `env:` and expands them correctly. The `env:` blocks themselves are also left in place; they are harmless and keep both call paths working. Files: - impl-review.yml: 4 lines (LIBRARY / SPEC_ID / PR_NUMBER / ATTEMPT header) - spec-create.yml: 8 lines (issue header in initial + retry prompts) - report-validate.yml: 4 lines (issue header)
Contributor
There was a problem hiding this comment.
Pull request overview
Restores correct GitHub Actions expression interpolation inside claude-code-action with: prompt: | blocks so runtime context (issue/PR/spec/library) is actually passed to Claude, avoiding ambiguous prompts that can lead to missing quality_score.txt outputs and downstream ai-review-failed results.
Changes:
- Update
spec-create.ymlClaude prompts to use${{ github.event.issue.* }}instead of literal$ISSUE_*placeholders. - Update
report-validate.ymlClaude prompt to use${{ github.event.issue.* }}instead of literal$ISSUE_*placeholders. - Update
impl-review.ymlClaude prompt “Variables for this run” header to use step outputs (${{ steps.*.outputs.* }}) instead of literal$LIBRARY/$SPEC_ID/...placeholders.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| .github/workflows/spec-create.yml | Fix prompt variable interpolation for issue details in both primary + retry Claude steps. |
| .github/workflows/report-validate.yml | Fix prompt variable interpolation for issue details in the Claude validation step. |
| .github/workflows/impl-review.yml | Fix prompt variable interpolation for library/spec/pr/attempt in the AI quality review step. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three workflows (
impl-review.yml,spec-create.yml,report-validate.yml) used shell-style$VARinsidewith: prompt: |blocks ofclaude-code-action. That block is a YAML string handed to a Node/Bun action — no shell ever runs, so$VARwas sent to Claude as a literal placeholder instead of the actual value. Result: Claude couldn't reliably identify the PR / spec / library to review and silently produced noquality_score.txt, which the validate step turns intoai-review-failed.Symptoms observed today (2026-04-29)
5 stuck implementation PRs from 2026-04-27, all with
ai-review-faileddespite the prior fixes branch (#5410) and the audit branch (#5515) landing in between:ai-review-failed,quality:78ai-review-failed,quality:82ai-rejected,quality:76ai-review-failedai-review-failedRe-dispatching review on each confirmed the bug: the run log of
Run AI Quality Reviewshows the prompt being passed verbatim:Claude's review then either ran for ~20s and exited with no
quality_score.txt(4 PRs failed), or recovered by inferring values from cwd (1 PR succeeded withquality:82). The intermittent pattern is exactly what you'd expect from "the prompt is ambiguous and Claude has to guess from context."Root cause
Commit
252977cf3("chore: fix critical audit findings", 2026-04-28 22:46) routed several${{ github.event.* }}and step-output values through step-levelenv:and rewrote the in-prompt references as$VAR. That is the correct mitigation forrun:shell steps and Python heredocs in the same workflows (and those changes stay in place). Insidewith: prompt: |it is the wrong tool: the value is consumed by a JS action, not a shell, so there is no injection surface to mitigate and$VARdoes not interpolate.spec-create.ymlandreport-validate.ymlcarry the identical anti-pattern in theirprompt:blocks. They haven't surfaced as failures yet only because no triggering issue has come in since 2026-04-28.The fix
Revert only the descriptive header lines of each
prompt:block back to GitHub Actions Expression syntax (${{ ... }}), which the runner substitutes into the YAML string before the action receives it. Keep:env:blocks (harmless; lets future prompt content reference env vars if useful)$VARreferences inside embedded bash code samples in the prompt (e.g.gh issue edit $ISSUE_NUMBER). Those are executed by Claude's Bash tool which inherits the stepenv:and expands them correctly — and rewriting them would re-enable the injection vector the audit was right to close.(analogous 8-line revert in
spec-create.yml× 2 prompt blocks and 4-line revert inreport-validate.yml).Diff total: 3 files, 16 ±.
Test plan
impl-review.ymlfor the 4 stuck PRs (gh workflow run impl-review.yml -f pr_number=<N>for 5476, 5483, 5486; 5480 already got a 82 in the redispatch and should now stabilize)Run AI Quality Reviewstep log shows real values (e.g.- LIBRARY: plotly) in the PROMPT echo, not$LIBRARYquality_score.txtis produced andai-review-failedlabel is removedspec-request-labeled issue, verify the spec-create prompt sees the issue title/bodyreport-pending-labeled issue, verify the report-validate prompt sees the issue title/body🤖 Generated with Claude Code