Skip to content

Make automatic per-module UI test runs blocking in CI - #50344

Merged
Gleb Khmyznikov (khmyznikov) merged 2 commits into
mainfrom
copilot/50220-update-ui-tests-blocking
Sep 5, 2026
Merged

Make automatic per-module UI test runs blocking in CI#50344
Gleb Khmyznikov (khmyznikov) merged 2 commits into
mainfrom
copilot/50220-update-ui-tests-blocking

Conversation

Copilot AI commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary of the Pull Request

Follow-up to #50220: the automatic per-module UI test jobs and their resolver step were non-blocking (continueOnError: true), so a failing auto-selected UI test — or a broken resolver — could not fail a PR build. This PR removes that coupling so both are blocking again, while preserving the "no affected UI tests" skip and the runAffectedUiTests kill switch.

Detailed Description of the Pull Request / Additional comments

job-test-project.yml

  • Removed continueOnError: ${{ parameters.autoSelectUiTests }} on the Test job — auto-selected UI test failures now fail the job.
  • PublishTestResults@2's failTaskOnFailedTests now follows ${{ parameters.autoSelectUiTests }} instead of a hardcoded false, so published failed tests fail the task on the auto path. The "Run UI Tests" pwsh step already exit 1s when any test assembly reports a non-zero exit code, so this was already the primary failure signal — this closes the remaining gap.

job-resolve-ui-tests.yml

  • Removed continueOnError: true on the Resolve job — a broken resolver now fails the build instead of silently skipping all downstream UI test jobs. Zero affected UI test projects (hasUiTests == 'false') remains a valid, successful skip; only resolver errors now surface.

pipeline-ci-build.yml / ci.yml

  • No changes needed: the auto UI test jobs' condition: and(succeeded(), eq(dependencies.Resolve.outputs['resolve.hasUiTests'], 'true')) already gates correctly on Resolve success, and there's no stage-level continueOnError masking failures.
  • The runAffectedUiTests pipeline parameter remains as the escape hatch — set runAffectedUiTests: false to disable the whole feature if it becomes flaky.

Validation Steps Performed

Reviewed the full job/stage/pipeline YAML chain (job-test-project.yml, job-resolve-ui-tests.yml, pipeline-ci-build.yml, ci.yml) to confirm no remaining continueOnError or condition allows a failed auto UI test or resolver run to be reported as succeeded.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: khmyznikov <6115884+khmyznikov@users.noreply.github.com>
Copilot AI changed the title [WIP] Update UI tests to be blocking on failure Make automatic per-module UI test runs blocking in CI Sep 3, 2026
@khmyznikov
Gleb Khmyznikov (khmyznikov) marked this pull request as ready for review September 3, 2026 03:23
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🧭 PR intake

Visual evidence: Not needed — The changed files do not indicate a visible UI change. No visual evidence is expected.

Recommendation

Link the issue this PR fixes using a closing keyword such as Closes #123.

✅ Ready for review

This PR passed the automated intake checks and is ready for maintainer review.

Automated PR intake; PowerToys maintainers make final decisions.

@khmyznikov
Gleb Khmyznikov (khmyznikov) merged commit 865ed8e into main Sep 5, 2026
11 of 13 checks passed
@khmyznikov
Gleb Khmyznikov (khmyznikov) deleted the copilot/50220-update-ui-tests-blocking branch September 5, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants