New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[flutter_tools] add shuffle to hermetic run_tests #105462
[flutter_tools] add shuffle to hermetic run_tests #105462
Conversation
@@ -431,7 +424,7 @@ void main() { | |||
expect(usage.commands, contains( | |||
TestUsageCommand('run', parameters: CustomDimensions.fromMap(<String, String>{ | |||
'cd3': 'false', 'cd4': 'ios', 'cd22': 'iOS 13', | |||
'cd23': 'debug', 'cd18': 'false', 'cd15': 'swift', 'cd31': 'false', | |||
'cd23': 'debug', 'cd18': 'false', 'cd15': 'swift', 'cd31': 'true', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cd31 is commandHasTerminal (https://github.com/christopherfujino/flutter/blob/improve-run-test/packages/flutter_tools/lib/src/reporting/custom_dimensions.dart#L102). The global environment was leaking into this test previously; thus, this test was passing on CI, since the Dart VM did not have a terminal attached, yet failing locally since it did. By overriding the context with a FakeStdio
that defaults to hasTerminal = true
, this dimension should be true both locally and on CI.
final MemoryIOSink _stderr = MemoryIOSink(); | ||
final StreamController<List<int>> _stdin = StreamController<List<int>>(); | ||
final FakeStdin _stdin = FakeStdin(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously this was failing the cast in https://github.com/christopherfujino/flutter/blob/improve-run-test/packages/flutter_tools/lib/src/base/terminal.dart#L286.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Part of #85160
Note, the fix to allowing shuffle was actually lifting Cache.disableLocking() to the top level scope in https://github.com/flutter/flutter/pull/105283/files#diff-38f3e3415d9806f22a78060fafb3206d1a006f74631fac04e7a680a64e06ccf0R52, however, I didn't actually disable shuffle in that PR because many other tests in that file were failing locally. The code changes in this PR are just to remove the local terminal from the test code, so that the tests are more hermetic (and all pass for me locally now).