Skip to content
This repository has been archived by the owner on Mar 3, 2023. It is now read-only.

Tests: retry timed out steps automatically #21295

Merged
merged 13 commits into from
Sep 18, 2020

Conversation

aminya
Copy link
Contributor

@aminya aminya commented Sep 10, 2020

Description of the change

This automatically detects the timed out tests and reruns them.

This is much faster than rerunning the tests manually in Azure. This is because the rerunning happens at a low JavaScript level compared to CI reruns that need to set up the whole environment and rerun all the tests just because one was timed out.

Verification

The CI passes. The functions are commented with details.

Release Notes

N/A

@aminya aminya changed the title Retry timed out steps automatically Tests: retry timed out steps automatically Sep 10, 2020
@aminya
Copy link
Contributor Author

aminya commented Sep 10, 2020

After using this PR in @atom-ide-community, I did not have any timed-out tests that require manual retrying. Only real genuine test failures show themselves. #21292

@sadick254
Copy link
Contributor

sadick254 commented Sep 10, 2020

@aminya This looks great. I like the retrying bits of this PR. Could you clean your commits to make it ready for review?
This might supersede #21292

@aminya
Copy link
Contributor Author

aminya commented Sep 10, 2020

@sadick254 If we merge #21269 and #21262, we will be only 1 PR away from parallelizing the tests in #21109. After that, this will be cleaned automatically.

It will be hard to make this independent of #21109, as I have changed the way the tests run in that PR.

@sadick254
Copy link
Contributor

@aminya needs rebasing

@aminya
Copy link
Contributor Author

aminya commented Sep 17, 2020

@sadick254 This is rebased! Ready to go.

If you want we can decrease the timeout interval instead of waiting for 12 seconds, and let our retry mechanism deal with it instead.

if process.env.CI
jasmine.getEnv().defaultTimeoutInterval = 120000
else
jasmine.getEnv().defaultTimeoutInterval = 5000

@aminya
Copy link
Contributor Author

aminya commented Sep 17, 2020

Also, another note: I haven't run prettier on this script (to keep the diff small). If you are done reviewing, we should do that.

@sadick254
Copy link
Contributor

If you want we can decrease the timeout interval instead of waiting for 12 seconds, and let our retry mechanism deal with it instead.

Lets leave it as it is for now.

Copy link
Contributor

@sadick254 sadick254 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aminya Thank you for your continuous contributions.

@sadick254 sadick254 merged commit 9cc29fa into atom:master Sep 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants