Make automatic per-module UI test runs blocking in CI - #50344
Merged
Gleb Khmyznikov (khmyznikov) merged 2 commits intoSep 5, 2026
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Co-authored-by: khmyznikov <6115884+khmyznikov@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Update UI tests to be blocking on failure
Make automatic per-module UI test runs blocking in CI
Sep 3, 2026
Gleb Khmyznikov (khmyznikov)
marked this pull request as ready for review
September 3, 2026 03:23
🧭 PR intakeVisual evidence: Not needed — The changed files do not indicate a visible UI change. No visual evidence is expected. Recommendation
✅ Ready for reviewThis PR passed the automated intake checks and is ready for maintainer review. Automated PR intake; PowerToys maintainers make final decisions. |
Gleb Khmyznikov (khmyznikov)
approved these changes
Sep 3, 2026
Niels Laute (niels9001)
approved these changes
Sep 4, 2026
Gleb Khmyznikov (khmyznikov)
deleted the
copilot/50220-update-ui-tests-blocking
branch
September 5, 2026 17:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of the Pull Request
Follow-up to #50220: the automatic per-module UI test jobs and their resolver step were non-blocking (
continueOnError: true), so a failing auto-selected UI test — or a broken resolver — could not fail a PR build. This PR removes that coupling so both are blocking again, while preserving the "no affected UI tests" skip and therunAffectedUiTestskill switch.Detailed Description of the Pull Request / Additional comments
job-test-project.ymlcontinueOnError: ${{ parameters.autoSelectUiTests }}on theTestjob — auto-selected UI test failures now fail the job.PublishTestResults@2'sfailTaskOnFailedTestsnow follows${{ parameters.autoSelectUiTests }}instead of a hardcodedfalse, so published failed tests fail the task on the auto path. The "Run UI Tests"pwshstep alreadyexit 1s when any test assembly reports a non-zero exit code, so this was already the primary failure signal — this closes the remaining gap.job-resolve-ui-tests.ymlcontinueOnError: trueon theResolvejob — a broken resolver now fails the build instead of silently skipping all downstream UI test jobs. Zero affected UI test projects (hasUiTests == 'false') remains a valid, successful skip; only resolver errors now surface.pipeline-ci-build.yml/ci.ymlcondition: and(succeeded(), eq(dependencies.Resolve.outputs['resolve.hasUiTests'], 'true'))already gates correctly onResolvesuccess, and there's no stage-levelcontinueOnErrormasking failures.runAffectedUiTestspipeline parameter remains as the escape hatch — setrunAffectedUiTests: falseto disable the whole feature if it becomes flaky.Validation Steps Performed
Reviewed the full job/stage/pipeline YAML chain (
job-test-project.yml,job-resolve-ui-tests.yml,pipeline-ci-build.yml,ci.yml) to confirm no remainingcontinueOnErroror condition allows a failed auto UI test or resolver run to be reported as succeeded.