Fix line filters running tests from multiple runnables. #23456
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`derive_regexp` was written with the assumption that we were run from a blank slate — that if the filter didn't match we might as well return it because it was nil. This isn't the case because minitest calls `run` on every runnable. Which is any subclass of Minitest::Runnable, such as ActiveSupport::TestCase, ActionDispatch::IntegrationTest as well as any inheriting from those. Thus after the first `run` we'd have put in a composite filter in `options[:filter]` making the next `run` create a linked list when it failed to match the regexp and put the composite filter as the head. Every runnable would accumulate more and more of the same filters, which effectively acted like an expanding whitelist and we ran tests from other runnables. Clog the accumulation by returning nil if there's no filter to derive a regexp from. Note: we pass a seed in the tests because Minitest shuffles the runnables to ensure the whitelist is expanded enough that the failure is triggered.
Because of the expanding whitelist for test filters, this test ended up running the tests on lines 4 and 9 in the post test even though the path wasn't right. Happened incidentally because the same line numbers were used in both account and post test. Add the .rb line so the file is required correctly and the filters are applied.
kaspth
added a commit
that referenced
this pull request
Feb 3, 2016
Fix line filters running tests from multiple runnables.
|
@spastorino here you go, you shouldn't see multiple tests with the same name run if you're passing in a line now 😄 |
|
@kaspth great job 😄 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
derive_regexpwas written with the assumption that we were run from ablank slate — that if the filter didn't match we might as well return it
because it was nil.
This isn't the case because minitest calls
runon every runnable. Whichis any subclass of Minitest::Runnable, such as ActiveSupport::TestCase,
ActionDispatch::IntegrationTest as well as any inheriting from those.
Thus after the first
runwe'd have put in a composite filter inoptions[:filter]making the nextruncreate a linked list when itfailed to match the regexp and put the composite filter as the head.
Every runnable would accumulate more and more of the same filters,
which effectively acted like an expanding whitelist and we ran tests
from other runnables.
Clog the accumulation by returning nil if there's no filter to derive
a regexp from.
Note: we pass a seed in the tests because Minitest shuffles the runnables
to ensure the whitelist is expanded enough that the failure is triggered.
r? @kaspth