Build flavor on iOS not working

[lsuhov]

Hi,

When running a simple integration test(without gherkin) with flutter driver --flavor=development --target=test_driver/app.dart, it works on android and iOS emulators.

I'm trying to integrate Gherkin, and I've set the FlutterTestConfiguration.buildFlavor to development. It works fine on Android, but fails on iOS.

Stacktrace:
DriverError: Failed to fulfill GetHealth due to remote error
Original error: JSON-RPC error -32601 (method not found): Method not found
Original stack trace:
package:json_rpc_2/src/client.dart 110:64 Client.sendRequest
package:json_rpc_2/src/peer.dart 79:15 Peer.sendRequest
package:vm_service_client/src/scope.dart 64:23 Scope.sendRequestRaw
package:vm_service_client/src/isolate.dart 361:19 VMIsolateRef.invokeExtension
package:flutter_driver/src/driver/driver.dart 438:62 FlutterDriver._sendCommand
package:flutter_driver/src/driver/driver.dart 472:34 FlutterDriver.checkHealth
package:flutter_driver/src/driver/driver.dart 361:29 FlutterDriver.connect.checkHealth
package:flutter_driver/src/driver/driver.dart 376:44 FlutterDriver.connect
===== asynchronous gap ===========================
package:flutter_gherkin/src/flutter/flutter_test_configuration.dart 118:34 FlutterTestConfiguration._attemptDriverConnection
package:flutter_gherkin/src/flutter/flutter_test_configuration.dart 75:18 FlutterTestConfiguration.createFlutterDriver
package:flutter_gherkin/src/flutter/flutter_test_configuration.dart 81:26 FlutterTestConfiguration.createFlutterWorld
package:flutter_gherkin/src/flutter/flutter_test_configuration.dart 95:20 FlutterTestConfiguration.prepare.
package:gherkin/src/feature_file_runner.dart 159:31 FeatureFileRunner._runScenario
===== asynchronous gap ===========================
package:gherkin/src/feature_file_runner.dart 118:30 FeatureFileRunner._runScenarioInZone.
dart:async/zone.dart 1126:13 _rootRun
dart:async/zone.dart 1023:19 _CustomZone.run
dart:async/zone.dart 1518:10 _runZoned
dart:async/zone.dart 1502:12 runZoned
package:gherkin/src/feature_file_runner.dart 116:5 FeatureFileRunner._runScenarioInZone
package:gherkin/src/feature_file_runner.dart 63:21 FeatureFileRunner._runFeature
===== asynchronous gap ===========================
package:gherkin/src/feature_file_runner.dart 36:38 FeatureFileRunner.run
package:gherkin/src/test_runner.dart 77:45 GherkinRunner.execute
===== asynchronous gap ===========================
test_driver/main_tests/test_config.dart 34:28 TestConfig.execute
test_driver/main_tests/app_dev_test.dart 7:21 main
dart:isolate-patch/isolate_patch.dart 307:19 _startIsolate.
dart:isolate-patch/isolate_patch.dart 174:12 _RawReceivePortImpl._handleMessage

Unhandled exception:
Bad state: No element
#0 List.last (dart:core-patch/growable_array.dart:227:5)
#1 JsonFeature.currentScenario (package:gherkin/src/reporters/json/json_feature.dart:33:22)
#2 JsonReporter.onException (package:gherkin/src/reporters/json/json_reporter.dart:40:10)
#3 AggregatedReporter.onException. (package:gherkin/src/reporters/aggregated_reporter.dart:58:30)
#4 AggregatedReporter._invokeReporters (package:gherkin/src/reporters/aggregated_reporter.dart:70:21)

#5 AggregatedReporter.onException (package:gherkin/src/reporters/aggregated_reporter.dart:57:11)
#6 FeatureFileRunner._runScenarioInZone. (package:gherkin/src/feature_file_runner.dart:131:23)
#7 _rootRunBinary (dart:async/zone.dart:1150:13)
#8 _RootZone.runBinary (dart:async/zone.dart:1387:12)
#9 runZoned. (dart:async/zone.dart:1481:21)
#10 _CustomZone.handleUncaughtError (dart:async/zone.dart:1005:19)
#11 Future._propagateToListeners (dart:async/future_impl.dart:596:16)
#12 Future._completeError (dart:async/future_impl.dart:532:5)
#13 Future.timeout. (dart:async/future_impl.dart:783:16)
#14 _rootRunBinary (dart:async/zone.dart:1146:38)
#15 _CustomZone.runBinary (dart:async/zone.dart:1039:19)
#16 _FutureListener.handleError (dart:async/future_impl.dart:153:20)
#17 Future._propagateToListeners.handleError (dart:async/future_impl.dart:692:47)
#18 Future._propagateToListeners (dart:async/future_impl.dart:713:24)
#19 Future._propagateToListeners (dart:async/future_impl.dart:607:9)
#20 Future._completeError (dart:async/future_impl.dart:532:5)
#21 _AsyncAwaitCompleter.completeError (dart:async-patch/async_patch.dart:38:15)
#22 Scope.sendRequestRaw (package:vm_service_client/src/scope.dart)

#23 VMIsolateRef.invokeExtension (package:vm_service_client/src/isolate.dart:361:19)
#24 FlutterDriver._sendCommand (package:flutter_driver/src/driver/driver.dart:438:62)
#25 FlutterDriver.checkHealth (package:flutter_driver/src/driver/driver.dart:472:34)
#26 FlutterDriver.connect.checkHealth (package:flutter_driver/src/driver/driver.dart:361:29)
#27 FlutterDriver.connect (package:flutter_driver/src/driver/driver.dart:376:44)

#28 FlutterTestConfiguration._attemptDriverConnection (package:flutter_gherkin/src/flutter/flutter_test_configuration.dart:118:34)
#29 FlutterTestConfiguration.createFlutterDriver (package:flutter_gherkin/src/flutter/flutter_test_configuration.dart:75:18)
#30 FlutterTestConfiguration.createFlutterWorld (package:flutter_gherkin/src/flutter/flutter_test_configuration.dart:81:26)
#31 FlutterTestConfiguration.prepare. (package:flutter_gherkin/src/flutter/flutter_test_configuration.dart:95:20)
#32 FeatureFileRunner._runScenario (package:gherkin/src/feature_file_runner.dart:159:31)

#33 FeatureFileRunner._runScenarioInZone. (package:gherkin/src/feature_file_runner.dart:118:30)
#34 _rootRun (dart:async/zone.dart:1126:13)
#35 _CustomZone.run (dart:async/zone.dart:1023:19)
#36 _runZoned (dart:async/zone.dart:1518:10)
#37 runZoned (dart:async/zone.dart:1502:12)
#38 FeatureFileRunner._runScenarioInZone (package:gherkin/src/feature_file_runner.dart:116:5)
#39 FeatureFileRunner._runFeature (package:gherkin/src/feature_file_runner.dart:63:21)

#40 FeatureFileRunner.run (package:gherkin/src/feature_file_runner.dart:36:38)
#41 GherkinRunner.execute (package:gherkin/src/test_runner.dart:77:45)

#42 TestConfig.execute (file:///Users/app/test_driver/main_tests/test_config.dart:34:28)
#43 main (file:///Users/app/test_driver/main_tests/app_dev_test.dart:7:21)
#44 _startIsolate. (dart:isolate-patch/isolate_patch.dart:307:19)
#45 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:174:12)

Flutter doctor:
[✓] Flutter (Channel stable, v1.12.13+hotfix.5, on Mac OS X 10.15.2 19C57, locale en-RO)
• Flutter version 1.12.13+hotfix.5
• Framework revision 27321ebbad (5 weeks ago), 2019-12-10 18:15:01 -0800
• Engine revision 2994f7e1e6
• Dart version 2.7.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.1)
• Android SDK at Library/Android/sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-29, build-tools 29.0.1
• ANDROID_HOME = Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
• All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 11.3.1, Build version 11C504
• CocoaPods version 1.7.5

[✓] Android Studio (version 3.5)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin version 42.1.1
• Dart plugin version 191.8593
• Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.41.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.7.1

[✓] Connected device (1 available)
• iPhone 11 Pro Max • ios • com.apple.CoreSimulator.SimRuntime.iOS-13-3 (simulator)

[jonsamwell]

Hi,

I am using build flavor successfully with ios and Android during Gherkin test runs. It must be your flavour setup in ios I would think.

Could you run the build with the property below set:

https://github.com/jonsamwell/flutter_gherkin/blob/master/lib/src/flutter/flutter_test_configuration.dart#L53

This should tell us the flavor passed to the flutter build process.

Do you have all the correct Debug-{FLAVOR} & Release-{FLAVOR} settings (like below) and the {FLAVOR}.xcscheme files?

Also the above stacktrace would indicate that the ios app has been built correctly and the Flutter Driver is attempting to connect.

[lsuhov]

Hello @jonsamwell ,
Thank you for the prompt response.

I'm having following flavours in project.pbxproj:
/* Begin XCConfigurationList section /
97C146E91CF9000F007C117D / Build configuration list for PBXProject "Runner" / = {
isa = XCConfigurationList;
buildConfigurations = (
97C147031CF9000F007C117D / Debug /,
291FD015239EE796006122C0 / Debug-production /,
291FD00D239EE6F5006122C0 / Debug-development /,
291FD011239EE747006122C0 / Debug-uat /,
97C147041CF9000F007C117D / Release /,
291FD00F239EE71B006122C0 / Release-development /,
291FD013239EE782006122C0 / Release-uat /,
291FD017239EE7AB006122C0 / Release-production /,
249021D3217E4FDB00AE95B9 / Profile /,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
97C147051CF9000F007C117D / Build configuration list for PBXNativeTarget "Runner" / = {
isa = XCConfigurationList;
buildConfigurations = (
97C147061CF9000F007C117D / Debug /,
291FD016239EE796006122C0 / Debug-production /,
291FD00E239EE6F5006122C0 / Debug-development /,
291FD012239EE747006122C0 / Debug-uat /,
97C147071CF9000F007C117D / Release /,
291FD010239EE71B006122C0 / Release-development /,
291FD014239EE782006122C0 / Release-uat /,
291FD018239EE7AB006122C0 / Release-production /,
249021D4217E4FDB00AE95B9 / Profile /,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
/ End XCConfigurationList section */

I'm also having flavours xcschemes. I tried to put the content of development scheme but the text it's not displayed here.

I've enabled the verboseFlutterProcessLogs flag but terminal buffer is limited. What should I look for in there?

Thank you

[jonsamwell]

Yes that all looks correct above.

Sorry I told you the wrong property to enable. Please enable logFlutterProcessOutput . See https://github.com/jonsamwell/flutter_gherkin#logFlutterProcessOutput

You should see the command the library invokes to run the app in the first couple of line in the output (like below)

[lsuhov]

Hello @jonsamwell ,

I can confirm that the following command is run:
Invoking from working directory ./ command: flutter run --target=test_driver/main_tests/app_dev.dart --flavor=development

I noticed that you are passing several parameters from command line in your screenshot. But I guess you are using them to create a more dynamic FlutterTestConfiguration.

How are you getting the --verbose command in flutter run?

[jonsamwell]

Hi @isuhov,

So is the correct flavor for ios being built now or are you still having the problem?

Yes I have been meaning to do a more real-world testing example with all the best practices stuff like the args I pass into the running, running the tests on build servers etc. I will try and find time soon. However, I use Args (https://pub.dev/packages/args) to pass in command line parameters that updates the test configuration. The parameters to the --verbose command in flutter is verboseFlutterProcessLogs

import 'dart:async';

import 'package:args/args.dart';
import 'package:easilog_auth/services/oauth_http_service.dart';
import 'package:easilog_core/easilog_core.dart';
import 'package:flutter_gherkin/flutter_gherkin.dart';
import 'package:gherkin/gherkin.dart';
import 'package:glob/glob.dart';
import 'package:logging/logging.dart';

import 'configurations/configuration.dart';
import 'configurations/configuration_dev.dart';
import 'configurations/configuration_local.dart';
import 'implementation/hooks/app_state_reset_hook.dart';
import 'implementation/parameters/random_email_parameter.dart';
import 'implementation/world/app_world.dart';

Future<void> main(List<String> arguments) {
  _setupLogging();

  final options = (ArgParser()
        ..addOption(
          'targetConfig',
          allowed: ['local', 'dev'],
          defaultsTo: 'dev',
        )
        ..addOption(
          'buildFlavor',
          allowed: ['development', 'production'],
          defaultsTo: 'development',
        )
        ..addOption('tagExpression')
        ..addOption(
          'buildTimeoutInSeconds',
          defaultsTo: '90',
        )
        ..addOption(
          'driverConnectionDelay',
          defaultsTo: '2',
        )
        ..addOption('device-id')
        ..addOption('device-name')
        ..addFlag(
          'build',
          negatable: true,
          defaultsTo: true,
        )
        ..addFlag(
          'logFlutterProcessOutput',
          negatable: true,
          defaultsTo: false,
        )
        ..addFlag(
          'verboseFlutterProcessLogs',
          negatable: true,
          defaultsTo: false,
        ))
      .parse(arguments);

  final config = FlutterTestConfiguration()
    ..defaultTimeout = const Duration(seconds: 60)
    ..features = [Glob(r"features/**.feature")]
    ..reporters = [
      ProgressReporter(),
      TestRunSummaryReporter(),
      JsonReporter(path: './report.json'),
      StdoutReporter(MessageLevel.warning),
      FlutterDriverReporter(),
    ]
    ..hooks = [
      AttachScreenshotOnFailedStepHook(),
      AppStateResetHook(),
    ]
    ..customStepParameterDefinitions = [
      RandomEmailParameter(),
    ]
    ..stepDefinitions = [
      // custom steps
    ]
    ..restartAppBetweenScenarios = false
    ..tagExpression = options['tagExpression'].toString()
    ..buildFlavor = options['buildFlavor'].toString()
    ..build = options['build'] as bool
    ..logFlutterProcessOutput = options['logFlutterProcessOutput'] as bool
    ..verboseFlutterProcessLogs = options['verboseFlutterProcessLogs'] as bool
    ..flutterBuildTimeout = Duration(
        seconds: int.parse(options['buildTimeoutInSeconds'].toString()))
    ..flutterDriverReconnectionDelay = Duration(
        seconds: int.parse(options['driverConnectionDelay'].toString()))
    ..targetAppPath =
        "tests/integration/test_app_${options['targetConfig']}.dart"
    ..targetAppWorkingDirecotry = "../../"
    ..targetDeviceId = options['device-id']?.toString()
    ..verboseFlutterProcessLogs = options['verboseFlutterProcessLogs'] as bool
    ..logFlutterProcessOutput = options['logFlutterProcessOutput'] as bool
    ..exitAfterTestRun = true
    ..createWorld = (TestConfiguration config) async => createAppWorld(
          options['targetConfig'].toString(),
          options['device-name']?.toString(),
        );

  return GherkinRunner().execute(config);
}

Future<World> createAppWorld(
  String targetEnvironment,
  String deviceName,
) async {
  final config = _getConfiguration(targetEnvironment);
  final oAuthHttpService = OAuthHttpService(
    config.services.apiUrl,
    TrustedCertifcateHttpClientFactory(() => true).factory,
    config.oauth.clientId,
    config.oauth.clientSecret,
  );

  final world = EasilogAppWorld(
    AuthenticationManager(oAuthHttpService),
    config,
  );

  return world.initialise('en_GB', 'yMd').map((_) => world).first;
}

Configuration _getConfiguration(String targetEnvironment) {
  if (targetEnvironment == 'dev') {
    return DevConfiguration();
  } else {
    return LocalConfiguration();
  }
}

void _setupLogging() {
  Logger.root.level = Level.ALL;
  Logger.root.onRecord.listen((LogRecord rec) {
    print('${rec.loggerName}: ${rec.level} - ${rec.message}');
  });
}

Then you can run it with various configurations from the command line

dart test_harness.dart --targetConfig dev --buildFlavor development --buildTimeoutInSeconds 180 --tagExpression "@debug" --verboseFlutterProcessLogs --logFlutterProcessOutput

[lsuhov]

Hello @jonsamwell ,

Unfortunately I'm still facing the same problem, but I will be checking next week if the build flavors are correctly set up in iOS. Will let you know then.

[jonsamwell]

Great, if the problem persists and you can either give me access to the code or create a sample that reproduces the errors I can help sort it.

[lsuhov]

Thank you for sharing the script. The part with args will be very helpful :)

[lsuhov]

Hello @jonsamwell.
I found the reason for my issue. It was due to a hardcoded FLUTTER_TARGET variable.

While implementing build flavours for iOS, in one of the steps it was saying that I should add .xcconfig files for each flavour. The content that I put in development.xcconfig was the following:

#include "Flutter/Generated.xcconfig"
#include "Pods/Target Support Files/Pods-Runner/Pods-Runner.release-development.xcconfig"
#include "Pods/Target Support Files/Pods-Runner/Pods-Runner.debug-development.xcconfig"

FLUTTER_TARGET=lib/main_dev.dart
bundle_suffix = .dev

That FLUTTER_TARGET was taken even when I was running integration tests. After removing it, tests started to run and even normal builds seem to be fine. So I'm not sure why it was recommended to put that variable.

Thank you for your support!

[jonsamwell]

Glad you got it working!