ci(docs-infra): enable the Selenium Promise Manager in e2e tests to a void flakiness on CI #39905
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aio/tests/e2e/protractor.conf.js
Outdated
| @@ -21,7 +21,12 @@ exports.config = { | |||
| }, | |||
| }, | |||
| directConnect: true, | |||
| SELENIUM_PROMISE_MANAGER: false, | |||
| // Keep he Selenium Promise Manager enabled to avoid flakiness on CI. | |||
There was a problem hiding this comment.
Suggested change
| // Keep he Selenium Promise Manager enabled to avoid flakiness on CI. | |
| // Keep the Selenium Promise Manager enabled to avoid flakiness on CI. |
| // See https://github.com/angular/angular/issues/39872 for more details. | ||
| // | ||
| // TODO(gkalpak): Set this back to `false` to align with CLI-generated apps when the flakiness is | ||
| // fixed in the future. |
There was a problem hiding this comment.
What? Flakiness can be randomly fixed "on its own". True story (see the first commit in this PR) 😆
petebacondarwin
approved these changes
Dec 1, 2020
|
You can preview fd15fd6 at https://pr39905-fd15fd6.ngbuilds.io/. |
In angular#32497, an option was introduced to retry failed docs examples tests a second time. This was done to work around some then recently introduced flakiness. After inspecting ~50 recent CI docs examples jobs, all tests passed on the first try. This commit gets rid of the retry attempt. This will avoid covering up any new flakiness introduces in the future.
…kiness on CI Since we turned off the Selenium Promise Manager in angular#39600, the AIO e2e tests have started flaking on CI. After trying out several things, the only change that seems to eliminate the flakiness is turning the Selenium Promise Manager back on (see angular#39873 for more details). This commit turns the Selenium Project Manager on to get rid of the flakiness. Fixes angular#39872
gkalpak
force-pushed
the
ci-aio-fix-e2e-flakiness
branch
from
fd15fd6
to
2ba3652
Compare
|
You can preview 2ba3652 at https://pr39905-2ba3652.ngbuilds.io/. |
gkalpak
added
action: merge
jessicajaniuk
pushed a commit
that referenced
this pull request
Dec 1, 2020
In #32497, an option was introduced to retry failed docs examples tests a second time. This was done to work around some then recently introduced flakiness. After inspecting ~50 recent CI docs examples jobs, all tests passed on the first try. This commit gets rid of the retry attempt. This will avoid covering up any new flakiness introduces in the future. PR Close #39905
jessicajaniuk
pushed a commit
that referenced
this pull request
Dec 1, 2020
…kiness on CI (#39905) Since we turned off the Selenium Promise Manager in #39600, the AIO e2e tests have started flaking on CI. After trying out several things, the only change that seems to eliminate the flakiness is turning the Selenium Promise Manager back on (see #39873 for more details). This commit turns the Selenium Project Manager on to get rid of the flakiness. Fixes #39872 PR Close #39905
jessicajaniuk
pushed a commit
that referenced
this pull request
Dec 1, 2020
…kiness on CI (#39905) Since we turned off the Selenium Promise Manager in #39600, the AIO e2e tests have started flaking on CI. After trying out several things, the only change that seems to eliminate the flakiness is turning the Selenium Promise Manager back on (see #39873 for more details). This commit turns the Selenium Project Manager on to get rid of the flakiness. Fixes #39872 PR Close #39905
|
This issue has been automatically locked due to inactivity. Read more about our automatic conversation locking policy. This action has been performed automatically by a bot. |
pullapprove
bot
added
area: dev-infra
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since we turned off the Selenium Promise Manager in #39600, the AIO e2e tests have started flaking on CI. After trying out several things, the only change that seems to eliminate the flakiness is turning the Selenium Promise Manager back on (see #39873 for more details).
This commit turns the Selenium Project Manager on to get rid of the flakiness.
Fixes #39872.