ci: don't fail "Publish test results" step when tests fail#675
Merged
Conversation
Agent-Logs-Url: https://github.com/TransactionProcessing/TransactionProcessorACL/sessions/056809f0-6df8-4568-b665-ab2dcea23543 Co-authored-by: StuartFerguson <16325469+StuartFerguson@users.noreply.github.com>
Up to standards ✅🟢 Issues
|
Copilot
AI
changed the title
[WIP] Update GitHub Actions workflow for test results reporting
ci: don't fail "Publish test results" step when tests fail
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dorny/test-reporterwas configured withfail-on-error: true, causing the "Publish test results" step to appear as the failing step even though the actual failure was already captured bydotnet test. The job correctly fails via the test execution steps — the reporter step adding a second failure is misleading noise.Change
.github/workflows/nightlybuild.yml: setfail-on-error: falseon thedorny/test-reporterstepdotnet teststill exits non-zero on test failure, failing the job as expected. The reporter step now always completes successfully and publishes results/annotations regardless of test outcome.Original prompt
Create a pull request to adjust the GitHub Actions workflow so that the "Publish test results" step does not fail when tests fail, while preserving that the overall job fails when any test command fails.
Context:
.github/workflows/nightlybuild.yml, the step "Publish test results" usesdorny/test-reporter@v1withfail-on-error: true, which causes the step to report failure when tests have failed.dotnet testbehavior unchanged), but the "Publish test results" step should not be marked as failed.Requested change:
.github/workflows/nightlybuild.ymlto setfail-on-error: falsefor the "Publish test results" step (or remove the option).Acceptance criteria:
if: always()so results are published even on failure.The following is the prior conversation context from the user's chat exploration (may be truncated):
User: Please find a solution for this failing job. Use the logs, job definition, and any referenced files where the failure occurred. Keep your response focused on the solution and include code suggestions when appropriate.
Assistant: The job is failing because an integration test (
RedeemVouchers) cannot obtain an OAuth token for a seeded merchant user:invalid_grant/"The username/password couple is invalid."tokenResponseResult.IsSuccesswas expectedTruebut wasFalsemerchantuser@testmerchant1.co.uk/123456merchantClientThis indicates the Security Service running in the integration-test docker stack does not recognize the credentials that the test believes it has just created (wrong password hashing/creation flow, wrong realm/tenant, wrong client, or the user isn’t actually being created in the correct environment).
Fix (most likely): ensure the integration tests create users with a password that matches the security service’s expected hashing/creation mechanism
Right now the test step “When I create the following security users” appears to create users, but the subsequent login fails as if the password was never set correctly. The most robust fix is to change the integration test user provisioning to set passwords via the same API path the Security Service uses, rather than directly inserting/bootstrapping users in a way that bypasses password hashing or policy.
Concrete code suggestion
In
TransactionProcessorACL.IntegrationTests/Steps/SharedSteps.cs(the step that creates security users), ensure the user creation call:Pseudo-shape (adapt to your actual API client):
If your security service requires an admin token to create users/set passwords, make sure the step acquires that admin token first and uses it for provisioning.
Fix (also common): align the client configuration used for password grant
The failing login uses client
"merchantClient". If the Security Service no longer allows Resource Owner Password Credentials (password grant) for that client (or requires different scopes/audience), the service will respond withinvalid_grant.Make sure the integration environment seeds a client that:
client_id/client_secretIf you recently upgraded identity components, consider switching the integration tests from password grant to a supported flow (e.g., client credentials for machine-to-machine calls, or a test-only auth bypass), but the quickest unblock is to restore the expected
"merchantClient"configuration in the integration test environment.Make the failure easier to diagnose (immediate improvement)
Update the assertion site so the test prints the token endpoint URL, HTTP status code, and response body (it already prints body, but ensure status code + endpoint ...
This pull request was created from Copilot chat.