CI: Fix "Publish Test Results" job to actually fail when tests fail#708
Closed
CI: Fix "Publish Test Results" job to actually fail when tests fail#708
Conversation
Contributor
Author
|
@jasonleenaylor Unfortunately, I hit an error while trying to use the custom Copilot setup steps configured for this repository and had to close this PR. The error I am seeing is: Once you or someone with the necessary access fixes the problem, please unassign and then reassign issue #0 to me and I'll retry. Thanks! |
Co-authored-by: jasonleenaylor <2295227+jasonleenaylor@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Analyze CI action failures during main merge
CI: Don't comment on closed PRs when push-to-main tests fail
Feb 25, 2026
Contributor
Author
|
@jasonleenaylor Unfortunately, I hit an error while trying to use the custom Copilot setup steps configured for this repository and had to close this PR. The error I am seeing is: Once you or someone with the necessary access fixes the problem, please unassign and then reassign issue #0 to me and I'll retry. Thanks! |
Co-authored-by: jasonleenaylor <2295227+jasonleenaylor@users.noreply.github.com>
Copilot
AI
changed the title
CI: Don't comment on closed PRs when push-to-main tests fail
CI: Fix "Publish Test Results" job to actually fail when tests fail
Feb 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
EnricoMi/publish-unit-test-result-actionwithfail_on: test failuresonly marks the "NUnit Tests" check run (created via the Checks API) as failed — it does not fail the GitHub Actions job step. The "Publish Test Results" job therefore always concludedsuccess, making it useless as a branch protection guard. PRs with failing tests could be merged unimpeded.Two changes to
CI.yml:comment_modeconditional — prevents the publish step from posting/updating comments on already-closed PRs when CI runs onpushto main (previouslyalways, nowofffor non-PR events)debug_build_and_testdid not succeed, so the "Publish Test Results" job actually fails and branch protection blocks the merge:No branch protection rule changes needed — the existing "Publish Test Results" requirement now works correctly.
CI-ready checklist
.github/commit-guidelines.md(subject ≤ 72 chars, no trailing punctuation; if body present, blank line then ≤ 80-char lines).Src/**folders touched, correspondingAGENTS.mdfiles are updated or explicitly confirmed still accurate.Notes for reviewers (optional)
The
fail_on: test failuressetting is kept — it still correctly marks the "NUnit Tests" check run as failed, which surfaces individual test failure annotations. The new explicit step is additive: it propagates the upstream job result to the "Publish Test Results" job conclusion so branch protection actually enforces passing tests.💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.
This change is