Run integration tests connected to a chrome device

[elliette]

Work towards #5703

Follow up to #5854

• Adds support for running integration tests connected to a chrome device
• Runs all supported tests connected to a chrome device during CI

The goal is to help catch regressions that are specific to web apps.

[elliette]

FYI @annagrin @polina-c

[kenzieschmoll]

why are we allowing target to be null now?

[elliette]

We need to parse the args to see if the testAppDevice arg is provided, but the parsing itself was asserting that the testTarget wasn't null. Therefore if we were trying to run all web tests (and therefore not specifying a test target), it would fail with the assertion error.

[kenzieschmoll]

I think what we want to do instead is leave this assertion here, and move the fix for this issue upstream to run_tests.dart

We can create the TestRunnerArgs from:

[
          ...testRunnerArgs,
          '${TestRunnerArgs.testTargetArg}$testTarget',
],

which is what we pass into the _runTest method in run_tests.dart. Instead of passing in a list of strings to that method, let's just pass in these args that you can parse at the top of the main method before you create the testAppDevice var. Then this assertion won't be hit, but we can still have the guarantee here that this code is not being called with a null test target

[elliette]

Not 100% sure the change I've made is what you asked for, but I added the assertion back and moved the check for whether the test supports the test app device from main to _runTest (so that we only do the check once we have a test target). LMK what you think!

[kenzieschmoll]

Added a couple suggestions below that would need to be paired with this:

  if (testTargetProvided) {
    final args = TestRunnerArgs(testRunnerArgs);
    // TODO(kenz): add support for specifying a directory as the target instead
    // of a single file.
    await _runTest(args);
  } else {
    // Run all tests since a target test was not provided.
    final testDirectory = Directory(_testDirectory);
    final testFiles = testDirectory
        .listSync(recursive: true)
        .where((testFile) => testFile.path.endsWith(_testSuffix));
    for (final testFile in testFiles) {
      final testTarget = testFile.path;
      final args = TestRunnerArgs([
        ...testRunnerArgs,
        '${TestRunnerArgs.testTargetArg}$testTarget',
      ]);
      await _runTest(args);
    }
  }

[elliette]

Oh nice! I think I've applied all the suggestions

[kenzieschmoll]

nit: use TestAppDevice.flutterTester.argName instead of the 'flutter-tester' string

[kenzieschmoll]

one nit and lgtm

[auto-submit]

auto label is removed for flutter/devtools, pr: 5917, due to - The status or check suite test_ddc has failed. Please fix the issues identified (or deflake) before re-applying this label.