ci: temporarily disable flaky VS Code extension E2E tests - #18413
Merged
Conversation
The VS Code extension E2E tests are flaky and currently failing in CI, blocking unrelated PRs. The zero-to-running shard intermittently fails in its "after each" teardown with an AggregateError on Windows (microsoft#18098), and because the extension_e2e_tests job gates CI, this fails runs on PRs that don't touch the extension. Temporarily disable the extension_e2e_tests job in tests.yml: - Force the job to always skip via `if: ${{ false && ... }}`, keeping the run_extension_e2e reference so the trigger-map binding test stays valid. - Remove the two aggregate-gate clauses (PR + non-PR) that treat a skipped e2e job as a failure, so the forced skip doesn't fail the overall gate. Extension unit tests (extension_tests_win, extension_bootstrap_linux) are unaffected and still run. Re-enabling is tracked by microsoft#18412. Refs microsoft#18098 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 18413Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 18413" |
Ankit Jain (radical)
marked this pull request as ready for review
June 22, 2026 23:10
Ankit Jain (radical)
enabled auto-merge (squash)
June 22, 2026 23:11
Contributor
There was a problem hiding this comment.
Pull request overview
This PR temporarily disables the flaky extension_e2e_tests job in .github/workflows/tests.yml. The zero-to-running E2E shard intermittently fails in its after each teardown with an AggregateError (tracked in #18098), and because the job gates CI, it was failing unrelated PRs. The job is force-skipped while keeping it wired into the workflow so the trigger-map binding test stays valid, and the aggregate-gate clauses that treat a skipped E2E job as a failure are removed. Re-enabling is tracked by #18412.
Changes:
- Force
extension_e2e_teststo always skip viaif: ${{ false && needs.setup_for_tests.outputs.run_extension_e2e == 'true' }}, retaining therun_extension_e2ereference so the binding test still passes. - Remove the PR and non-PR aggregate-gate clauses that fail CI when
extension_e2e_testsis skipped. - Update surrounding comments to explain the temporary disable and how to revert.
David Fowler (davidfowl)
approved these changes
Jun 22, 2026
Contributor
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
Ankit Jain (radical)
merged commit Jun 22, 2026
d619e31
into
microsoft:main
643 of 647 checks passed
David Fowler (davidfowl)
pushed a commit
that referenced
this pull request
Jun 29, 2026
The extension E2E shards (specifically Windows, debug-dashboard) are consistently failing on main, blocking unrelated PRs from merging. Recent failures on main (last 5 days): - CI #25637 (Jun 28): Update OpenTelemetry.Instrumentation.StackExchangeRedis - CI #25621 (Jun 27): Prepare VS Code extension release v1.16.0 - CI #25616 (Jun 27): Fix VS Code extension CLI install - CI #25543 (Jun 26): Make dashboard launch opt-in by default Adds a `false &&` guard to the extension_e2e_tests job condition and removes the corresponding gate skip-checks, following the same pattern as PR #18413. Tracked by #18412.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The VS Code extension E2E tests are flaky and currently failing in CI, blocking unrelated PRs. The
zero-to-runningshard intermittently fails in itsafter eachteardown:(tracked in #18098). Because the
extension_e2e_testsjob gates CI, this flakiness fails runs on PRs that don't touch the extension.The fix
Temporarily disable the
extension_e2e_testsjob intests.yml:if: ${{ false && … }}(keeping therun_extension_e2ereference so the trigger-map binding test stays valid).Extension unit tests (
extension_tests_win,extension_bootstrap_linux) are unaffected and still run.Re-enabling
Tracked by #18412. Revert: drop the
false &&guard and restore the two gate clauses, after the flakiness in #18098 is fixed.Refs #18098