Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: retry timed out steps automatically #21295

Merged
merged 13 commits into from Sep 18, 2020

Conversation

aminya
Copy link
Contributor

@aminya aminya commented Sep 10, 2020

Description of the change

This automatically detects the timed out tests and reruns them.

This is much faster than rerunning the tests manually in Azure. This is because the rerunning happens at a low JavaScript level compared to CI reruns that need to set up the whole environment and rerun all the tests just because one was timed out.

Verification

The CI passes. The functions are commented with details.

Release Notes

N/A

@aminya aminya changed the title Retry timed out steps automatically Tests: retry timed out steps automatically Sep 10, 2020
@aminya
Copy link
Contributor Author

aminya commented Sep 10, 2020

After using this PR in @atom-ide-community, I did not have any timed-out tests that require manual retrying. Only real genuine test failures show themselves. #21292

@sadick254
Copy link
Contributor

sadick254 commented Sep 10, 2020

@aminya This looks great. I like the retrying bits of this PR. Could you clean your commits to make it ready for review?
This might supersede #21292

@aminya
Copy link
Contributor Author

aminya commented Sep 10, 2020

@sadick254 If we merge #21269 and #21262, we will be only 1 PR away from parallelizing the tests in #21109. After that, this will be cleaned automatically.

It will be hard to make this independent of #21109, as I have changed the way the tests run in that PR.

@aminya aminya force-pushed the timed-out-tests-4upstream branch 3 times, most recently from 3ff2aa7 to f231685 Compare Sep 16, 2020
@sadick254
Copy link
Contributor

sadick254 commented Sep 16, 2020

@aminya needs rebasing

@aminya aminya force-pushed the timed-out-tests-4upstream branch from f231685 to 7d14d56 Compare Sep 17, 2020
@aminya
Copy link
Contributor Author

aminya commented Sep 17, 2020

@sadick254 This is rebased! Ready to go.

If you want we can decrease the timeout interval instead of waiting for 12 seconds, and let our retry mechanism deal with it instead.

if process.env.CI
jasmine.getEnv().defaultTimeoutInterval = 120000
else
jasmine.getEnv().defaultTimeoutInterval = 5000

@aminya
Copy link
Contributor Author

aminya commented Sep 17, 2020

Also, another note: I haven't run prettier on this script (to keep the diff small). If you are done reviewing, we should do that.

@sadick254
Copy link
Contributor

sadick254 commented Sep 18, 2020

If you want we can decrease the timeout interval instead of waiting for 12 seconds, and let our retry mechanism deal with it instead.

Lets leave it as it is for now.

Copy link
Contributor

@sadick254 sadick254 left a comment

@aminya Thank you for your continuous contributions.

@sadick254 sadick254 merged commit 9cc29fa into atom:master Sep 18, 2020
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants