-
Notifications
You must be signed in to change notification settings - Fork 17.4k
Tests: retry timed out steps automatically #21295
Tests: retry timed out steps automatically #21295
Conversation
After using this PR in @atom-ide-community, I did not have any timed-out tests that require manual retrying. Only real genuine test failures show themselves. #21292 |
@sadick254 If we merge #21269 and #21262, we will be only 1 PR away from parallelizing the tests in #21109. After that, this will be cleaned automatically. It will be hard to make this independent of #21109, as I have changed the way the tests run in that PR. |
3ff2aa7
to
f231685
Compare
@aminya needs rebasing |
Don't call the callback on retry
don't finalize until the end which the test fails or passes.
f231685
to
7d14d56
Compare
@sadick254 This is rebased! Ready to go. If you want we can decrease the timeout interval instead of waiting for 12 seconds, and let our retry mechanism deal with it instead. Lines 55 to 58 in 7a2cc45
|
Also, another note: I haven't run prettier on this script (to keep the diff small). If you are done reviewing, we should do that. |
Lets leave it as it is for now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@aminya Thank you for your continuous contributions.
Description of the change
This automatically detects the timed out tests and reruns them.
This is much faster than rerunning the tests manually in Azure. This is because the rerunning happens at a low JavaScript level compared to CI reruns that need to set up the whole environment and rerun all the tests just because one was timed out.
Verification
The CI passes. The functions are commented with details.
Release Notes
N/A