Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(test): add --test-randomize-ordering-seed #326

Merged
merged 2 commits into from
Mar 18, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,14 +161,15 @@ very_good test -r
Run tests in a Dart or Flutter project.

Usage: very_good test [arguments]
-h, --help Print this usage information.
--coverage Whether to collect coverage information.
-r, --recursive Run tests recursively for all nested packages.
--[no-]optimization Whether to apply optimizations for test performance.
(defaults to on)
--exclude-coverage A glob which will be used to exclude files that match from the coverage.
-x, --exclude-tags Run only tests that do not have the specified tags.
--min-coverage Whether to enforce a minimum coverage percentage.
-h, --help Print this usage information.
--coverage Whether to collect coverage information.
-r, --recursive Run tests recursively for all nested packages.
--[no-]optimization Whether to apply optimizations for test performance.
(defaults to on)
--exclude-coverage A glob which will be used to exclude files that match from the coverage.
-x, --exclude-tags Run only tests that do not have the specified tags.
--min-coverage Whether to enforce a minimum coverage percentage.
--test-randomize-ordering-seed The seed to randomize the execution order of test cases within test files.

Run "very_good help" to see global options.
```
Expand Down
11 changes: 11 additions & 0 deletions lib/src/commands/test/test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ class TestCommand extends Command<int> {
..addOption(
'min-coverage',
help: 'Whether to enforce a minimum coverage percentage.',
)
..addOption(
'test-randomize-ordering-seed',
help: 'The seed to randomize the execution order of test cases '
'within test files.',
);
}

Expand Down Expand Up @@ -106,6 +111,8 @@ This command should be run from the root of your Flutter project.''',
final excludeTags = _argResults['exclude-tags'] as String?;
final isFlutterInstalled = await _flutterInstalled();
final excludeFromCoverage = _argResults['exclude-coverage'] as String?;
final randomOrderingSeed =
_argResults['test-randomize-ordering-seed'] as String?;
final optimizePerformance = _argResults['optimization'] as bool;

if (isFlutterInstalled) {
Expand All @@ -121,6 +128,10 @@ This command should be run from the root of your Flutter project.''',
excludeFromCoverage: excludeFromCoverage,
arguments: [
if (excludeTags != null) ...['-x', excludeTags],
if (randomOrderingSeed != null) ...[
'--test-randomize-ordering-seed',
randomOrderingSeed
],
'--no-pub',
..._argResults.rest,
],
Expand Down
60 changes: 52 additions & 8 deletions test/src/commands/test/test_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,15 @@ const expectedTestUsage = [
'Run tests in a Dart or Flutter project.\n'
'\n'
'Usage: very_good test [arguments]\n'
'-h, --help Print this usage information.\n'
' --coverage Whether to collect coverage information.\n'
'''-r, --recursive Run tests recursively for all nested packages.\n'''
''' --[no-]optimization Whether to apply optimizations for test performance.\n'''
' (defaults to on)\n'
''' --exclude-coverage A glob which will be used to exclude files that match from the coverage.\n'''
'''-x, --exclude-tags Run only tests that do not have the specified tags.\n'''
''' --min-coverage Whether to enforce a minimum coverage percentage.\n'''
'-h, --help Print this usage information.\n'
''' --coverage Whether to collect coverage information.\n'''
'''-r, --recursive Run tests recursively for all nested packages.\n'''
''' --[no-]optimization Whether to apply optimizations for test performance.\n'''
' (defaults to on)\n'
''' --exclude-coverage A glob which will be used to exclude files that match from the coverage.\n'''
'''-x, --exclude-tags Run only tests that do not have the specified tags.\n'''
''' --min-coverage Whether to enforce a minimum coverage percentage.\n'''
''' --test-randomize-ordering-seed The seed to randomize the execution order of test cases within test files.\n'''
'\n'
'Run "very_good help" to see global options.'
];
Expand Down Expand Up @@ -177,6 +178,49 @@ void main() {
).called(1);
});

test('completes normally --test-randomize-ordering-seed random', () async {
when<dynamic>(
() => argResults['test-randomize-ordering-seed'],
).thenReturn('random');
final result = await testCommand.run();
expect(result, equals(ExitCode.success.code));
verify(
() => flutterTest(
arguments: [
'--test-randomize-ordering-seed',
'random',
felangel marked this conversation as resolved.
Show resolved Hide resolved
...defaultArguments
],
optimizePerformance: true,
progress: logger.progress,
stdout: logger.write,
stderr: logger.err,
),
).called(1);
});

test('completes normally --test-randomize-ordering-seed 2305182648',
() async {
when<dynamic>(
() => argResults['test-randomize-ordering-seed'],
).thenReturn('2305182648');
final result = await testCommand.run();
expect(result, equals(ExitCode.success.code));
verify(
() => flutterTest(
arguments: [
'--test-randomize-ordering-seed',
'2305182648',
...defaultArguments
],
optimizePerformance: true,
progress: logger.progress,
stdout: logger.write,
stderr: logger.err,
),
).called(1);
});

test('completes normally --coverage', () async {
when<dynamic>(() => argResults['coverage']).thenReturn(true);
final result = await testCommand.run();
Expand Down