spec_helper: fix and improve retry logic.#10439
Merged
MikeMcQuaid merged 4 commits intoHomebrew:masterfrom Jan 27, 2021
MikeMcQuaid:fix_retry
Merged
spec_helper: fix and improve retry logic.#10439MikeMcQuaid merged 4 commits intoHomebrew:masterfrom MikeMcQuaid:fix_retry
MikeMcQuaid merged 4 commits intoHomebrew:masterfrom
MikeMcQuaid:fix_retry
Conversation
MikeMcQuaid
added
the
critical
Contributor
|
Review period skipped due to |
BrewTestBot
approved these changes
Jan 27, 2021
- always retry each test at least once (confusingly this means a retry count of 2 rather than 1) - always wait at least 1 second between retries - set a default retry metadata for integration tests rather than overriding any specified values - use `example.run` rather than `example.run_with_retry` for integration tests because, confusingly, this avoids having the retry count be half what it should be (because the attempts increases by one for each `run_with_retry` call) - use 4 retries for integration tests with 2**attempts*retry_wait (retry wait now being 2) to actually have more retries than before this commit (but with exponential backoff) This should generally improve test flakiness in CI but particularly improve the cleanup test flake we've seen recently.
The `VERBOSE_TESTS` variable was from cask and never gets set (and is unset by `bin/brew`). Replace it with `HOMEBREW_VERBOSE_TESTS` and set it by `--verbose` or `--debug`. While we're here, remove an unneeded `VERBOSE` delete (as it's already done by `bin/brew`).
It's annoying to have these autoremoved.
Let's see if this makes the test more reliable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
example.runrather thanexample.run_with_retryfor integration tests because, confusingly, this avoids having the retry count be half what it should be (because the attempts increases by one for eachrun_with_retrycall)This should generally improve test flakiness in CI but particularly improve the cleanup test flake we've seen recently.