Skip to content

Flakey tests#64

Merged
joshpencheon merged 7 commits intomasterfrom
flakey_tests
Feb 10, 2020
Merged

Flakey tests#64
joshpencheon merged 7 commits intomasterfrom
flakey_tests

Conversation

@joshpencheon
Copy link
Contributor

Adds a flakey_test method to minitest's DSL. Positioning as a pragmatic necessary evil, given CI pipelines are being affected.

If a test declared as "flakey" fails, but then passes after retries, it will be reported as a Pass. However, the CI will additional report it as being flakey (in a purple bubble to Slack).

Minitest appears to follow semantic versioning; pins to the 5.x release series.
@joshpencheon
Copy link
Contributor Author

I can't think of a good way to add independent test coverage of this, without having a separate test suite which the main test suite could run, and capture output from. The issue of keeping the tests separate stems from how minitest's Runnable uses the inherited callback to accumulate test definitions.

@joshpencheon joshpencheon merged commit a2f1788 into master Feb 10, 2020
@joshpencheon joshpencheon deleted the flakey_tests branch January 3, 2021 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant