Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --watch mode to flutter test (Redo) #27192

Closed
wants to merge 28 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
0a2e231
Pull test compiler out of FlutterPlatform
gamebox Jan 9, 2019
7188fc1
Pull test bootstrap generation out of FlutterPlatform
gamebox Jan 9, 2019
cff3e7f
Move test compilation to test runner from flutter_platform
gamebox Jan 9, 2019
98f0de0
Add --watch flag to flutter test
gamebox Jan 9, 2019
f916e6f
Update runner for upstream changes to test_core API
gamebox Jan 11, 2019
b9922db
Updates for new dart-lang/test release
gamebox Jan 24, 2019
3d5cbff
Update test dependency
gamebox Jan 24, 2019
5f73e36
Delete pubspec.lock for flutter_tools
gamebox Jan 24, 2019
411ae2e
Address review feedback
gamebox Jan 25, 2019
9fd7c0e
Import from path, not flutter_tools package
gamebox Jan 25, 2019
8089386
Remove trailing whitespace
gamebox Jan 25, 2019
548528a
Update test runner for changes to ANSI Terminal
gamebox Jan 28, 2019
2fa1d60
Upgrade packages
gamebox Jan 28, 2019
785eab4
Ignore dart:convert import in test listener file
gamebox Jan 28, 2019
ba2bfe6
Add devicelab tests for --watch flag
gamebox Jan 30, 2019
c3ceaae
Now with actual devicelab test
gamebox Jan 30, 2019
3d23b37
Address all review feedback
gamebox Feb 6, 2019
f149d74
More review feedback
gamebox Feb 7, 2019
fd8eb92
Fixes for test failures
gamebox Feb 7, 2019
ce3c998
Add protection for user holding 'r' in watch mode
gamebox Feb 8, 2019
6317fcc
Move watch flag mismatch checks to validateCommand()
gamebox Feb 8, 2019
4b4a750
Clear invalidated files after tests compile
gamebox Feb 8, 2019
4be6b18
Only compile if some files have been changed.
gamebox Feb 11, 2019
421354d
Update packages to resolve conflicts
gamebox Feb 11, 2019
0e328a1
Resolve pubspec conflicts
gamebox Feb 12, 2019
29165fb
Merge branch 'master' of github.com:flutter/flutter into ftw
gamebox Feb 12, 2019
5ae9e4e
Fix pubspec issues
gamebox Feb 12, 2019
b7679b8
Ignore package:analyze deprecation for now
gamebox Feb 12, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions dev/automated_tests/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ dependencies:
sdk: flutter
test: 1.5.3

analyzer: 0.33.6+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
analyzer: 0.35.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
args: 1.5.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
async: 2.0.8 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
boolean_selector: 1.0.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand All @@ -20,7 +20,7 @@ dependencies:
convert: 2.1.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
crypto: 2.0.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
csslib: 0.14.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
glob: 1.1.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
html: 0.13.3+3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http: 0.12.0+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand All @@ -29,7 +29,7 @@ dependencies:
io: 0.3.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
js: 0.6.1+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
json_rpc_2: 2.0.9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
logging: 0.11.3+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
matcher: 0.12.3+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
meta: 1.1.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand Down Expand Up @@ -68,4 +68,4 @@ dependencies:
flutter:
uses-material-design: true

# PUBSPEC CHECKSUM: 25f2
# PUBSPEC CHECKSUM: 1d20
8 changes: 4 additions & 4 deletions dev/benchmarks/complex_layout/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,19 @@ dev_dependencies:
sdk: flutter
test: 1.5.3

analyzer: 0.33.6+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
analyzer: 0.35.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
args: 1.5.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
boolean_selector: 1.0.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
csslib: 0.14.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
glob: 1.1.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
html: 0.13.3+3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http: 0.12.0+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_multi_server: 2.0.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_parser: 3.1.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
io: 0.3.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
js: 0.6.1+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
logging: 0.11.3+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
matcher: 0.12.3+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
mime: 0.9.6+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand Down Expand Up @@ -85,4 +85,4 @@ flutter:
- packages/flutter_gallery_assets/people/square/ali.png
- packages/flutter_gallery_assets/places/india_chettinad_silk_maker.png

# PUBSPEC CHECKSUM: 39c3
# PUBSPEC CHECKSUM: e3f0
8 changes: 4 additions & 4 deletions dev/benchmarks/macrobenchmarks/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,19 @@ dev_dependencies:
sdk: flutter
test: 1.5.3

analyzer: 0.33.6+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
analyzer: 0.35.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
args: 1.5.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
boolean_selector: 1.0.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
csslib: 0.14.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
glob: 1.1.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
html: 0.13.3+3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http: 0.12.0+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_multi_server: 2.0.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_parser: 3.1.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
io: 0.3.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
js: 0.6.1+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
logging: 0.11.3+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
matcher: 0.12.3+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
mime: 0.9.6+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand Down Expand Up @@ -82,4 +82,4 @@ dev_dependencies:
flutter:
uses-material-design: true

# PUBSPEC CHECKSUM: 39c3
# PUBSPEC CHECKSUM: e3f0
12 changes: 6 additions & 6 deletions dev/benchmarks/microbenchmarks/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ dependencies:
path: ../../../examples/stocks
test: 1.5.3

analyzer: 0.33.6+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
analyzer: 0.35.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
args: 1.5.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
async: 2.0.8 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
boolean_selector: 1.0.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand All @@ -24,20 +24,20 @@ dependencies:
convert: 2.1.1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
crypto: 2.0.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
csslib: 0.14.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
dart_style: 1.2.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
dart_style: 1.2.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
glob: 1.1.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
html: 0.13.3+3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http: 0.12.0+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_multi_server: 2.0.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_parser: 3.1.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
intl: 0.15.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
intl_translation: 0.17.2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
intl_translation: 0.17.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
io: 0.3.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
isolate: 2.0.2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
js: 0.6.1+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
json_rpc_2: 2.0.9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
logging: 0.11.3+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
matcher: 0.12.3+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
mime: 0.9.6+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand Down Expand Up @@ -76,4 +76,4 @@ dependencies:
flutter:
uses-material-design: true

# PUBSPEC CHECKSUM: fd49
# PUBSPEC CHECKSUM: 3b7a
8 changes: 4 additions & 4 deletions dev/bots/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ dependencies:
http_parser: 3.1.3
test: 1.5.3

analyzer: 0.33.6+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
analyzer: 0.35.0 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
async: 2.0.8 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
boolean_selector: 1.0.4 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
charcode: 1.1.2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand All @@ -24,15 +24,15 @@ dependencies:
crypto: 2.0.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
csslib: 0.14.6 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
file: 5.0.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
front_end: 0.1.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
glob: 1.1.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
html: 0.13.3+3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
http_multi_server: 2.0.5 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
intl: 0.15.7 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
io: 0.3.3 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
js: 0.6.1+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
json_rpc_2: 2.0.9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.6+9 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
kernel: 0.3.10 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
logging: 0.11.3+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
matcher: 0.12.3+1 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
mime: 0.9.6+2 # THIS LINE IS AUTOGENERATED - TO UPDATE USE "flutter update-packages --force-upgrade"
Expand Down Expand Up @@ -67,4 +67,4 @@ dev_dependencies:
mockito: 4.0.0
test_api: 0.2.2

# PUBSPEC CHECKSUM: ca97
# PUBSPEC CHECKSUM: 02c4
215 changes: 215 additions & 0 deletions dev/devicelab/bin/tasks/flutter_test__watch_test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
// Copyright (c) 2019 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

// This test runs `flutter test --watch` on the `trivial_widget_test.dart`
// three times.
//
// The intial run should pass.
//
// We then modify the source file, and the second time the test should fail.
//
// Before the third time, a change is made to test file, which should cause the
// third run to again pass.
//
// We then reset the files to their initial state.

import 'dart:async';
import 'dart:convert';
import 'dart:io';

import 'package:path/path.dart' as path;

import 'package:flutter_devicelab/framework/framework.dart';
import 'package:flutter_devicelab/framework/utils.dart';

const String rerunLine = "Press 'r' to rerun your tests, 'q' to quit";

class TestRunnerProcess {
TestRunnerProcess(this._process) {
_waitingForLine = rerunLine;
_completer = Completer<List<String>>();
_process.stdout.transform(utf8.decoder).transform(const LineSplitter()).listen((String line) {
_stdout.add(line);
if (_waitingForLine != null && line.contains(_waitingForLine)) {
print('Found line "$_waitingForLine", returning ${_stdout.length} lines');
_completer.complete(_stdout.map((String a) => a).toList());
_stdout.clear();
_waitingForLine = null;
}
});
}

final List<String> _stdout = <String>[]; // Collected lines
String _waitingForLine; // Substring of line currently waiting for
final Process _process; // Underlying system process
Completer<List<String>> _completer; // Used to control execution flow

Future<List<String>> sendChar(String char) {
_waitingForLine = rerunLine;
_completer = Completer<List<String>>();
_process.stdin.write(char[0]);
return _completer.future;
}

Future<List<String>> waitForLine(String line) {
_waitingForLine = line;
_completer = Completer<List<String>>();
return _completer.future;
}
}

Future<TestRunnerProcess> startTestRunner() async {
final Process process = await startProcess(
path.join(flutterDirectory.path, 'bin', 'flutter'),
<String>[
'test',
'-v',
'--watch',
path.join('flutter_test', 'trivial_widget_test.dart')
],
workingDirectory: path.join(flutterDirectory.path, 'dev', 'automated_tests'),
);

final TestRunnerProcess testRunner = TestRunnerProcess(process);
await testRunner.waitForLine(rerunLine);
return testRunner;
}

// This will go through the `actual` list in order, looking for matches to the
// `first` element of the expected list(contains), removing that element if found.
// If at the end of the actual list, expected has any elements left, it is an
// error.
TaskResult expectMatchesInOrder(List<String> actual, List<String> expected) {
if (expected.isEmpty) {
return TaskResult.success(<String, dynamic>{});
}
for (String line in actual) {
if (line.contains(expected.first)) {
expected.removeAt(0);
if (expected.isEmpty) {
break;
}
}
}

if (expected.isNotEmpty) {
return TaskResult.failure(
"Could not find '${expected.first}' in:\n${actual.join('\n')}"
);
}

return TaskResult.success(<String, dynamic>{});
}

Future<TaskResult> runTest(TestRunnerProcess testRunner, {
String description = 'A test',
List<String> stdoutExpected = const <String>[],
}) async {
print((description + ' ').padRight(80, '='));
final List<String> testRunOutput = await testRunner.sendChar('r');
return expectMatchesInOrder(testRunOutput, stdoutExpected);
}

void main() {
task(() async {
final File nodeSourceFile = File(path.join(
flutterDirectory.path, 'packages', 'flutter', 'lib', 'src', 'foundation', 'node.dart',
));
final File testSourceFile = File(path.join(
flutterDirectory.path, 'dev', 'automated_tests', 'flutter_test', 'trivial_widget_test.dart',
));

final String originalSource = await nodeSourceFile.readAsString();
final String testOriginalSource = await testSourceFile.readAsString();

try {
final TestRunnerProcess testRunner = await startTestRunner();
TaskResult result;

result = await runTest(testRunner,
description: 'Rerun with no change',
stdoutExpected: <String>[
'A trivial widget test',
'All tests passed!',
"Press 'r' to rerun",
]
);

if (!result.succeeded) {
return result;
}

await nodeSourceFile.writeAsString( // only change implementation
originalSource
.replaceAll('_owner', '_xyzzy')
);
result = await runTest(testRunner,
description: 'Rerun with implementation change to source',
stdoutExpected: <String>[
'Recompiling test files...',
'A trivial widget test',
'All tests passed!',
"Press 'r' to rerun",
],
);

if (!result.succeeded) {
return result;
}

await nodeSourceFile.writeAsString( // change interface as well
originalSource
.replaceAll('_owner', '_xyzzy')
.replaceAll('owner', '_owner')
.replaceAll('_xyzzy', 'owner')
);
result = await runTest(testRunner,
description: 'Rerun with interface changed in source',
stdoutExpected: <String>[
'Recompiling test files...',
'A trivial widget test',
'All tests passed!',
"Press 'r' to rerun",
],
);

if (!result.succeeded) {
return result;
}

// Test that introducing an error to the test source, creates a failure.
final String patchedTestSource =
testOriginalSource
.replaceAll(
'(WidgetTester tester) async {}',
'''
(WidgetTester tester) async {
expect(true, false);
}''');
final Future<List<String>> invalidateFuture = testRunner.waitForLine('Will invalidate');
await testSourceFile.writeAsString(patchedTestSource);
final List<String> invalidateOutput = await invalidateFuture;
final TaskResult invalidateResult = expectMatchesInOrder(invalidateOutput, <String>['Will invalidate']);

if (!invalidateResult.succeeded) {
return invalidateResult;
}

result = await runTest(testRunner,
description: 'Rerun with error in test file ',
stdoutExpected: <String>[
'Recompiling test files...',
'Expected: <false>',
'Actual: <true>',
'Some tests failed.',
],
);

return result;
} finally {
await nodeSourceFile.writeAsString(originalSource);
await testSourceFile.writeAsString(testOriginalSource);
}
});
}
7 changes: 7 additions & 0 deletions dev/devicelab/manifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,13 @@ tasks:
stage: devicelab
required_agent_capabilities: ["linux/android"]

flutter_test__watch_test:
description: >
Ensures that --watch flag to flutter test command works as intended.
stage: devicelab
required_agent_capabilities: ["linux/android"]
gamebox marked this conversation as resolved.
Show resolved Hide resolved
flaky: true

technical_debt__cost:
description: >
Estimates our technical debt (TODOs, analyzer ignores, etc).
Expand Down
Loading