enable xctest in .cirrus.yml #3189

cyanglaz · 2020-10-22T17:57:45Z

Enable XCTest in cirrus.

This PR will be merged to the xctest branch first.

As it for now, it skips all the plugins.

Presubmit xctests was successful in #3188, which tests a UITest in share plugin.

I wanted to land this to test post submit, especially testing if skipping works properly.

amirh · 2020-10-22T18:00:47Z

.cirrus.yml

+        plugins_to_skip: "battery/battery,camera,connectivity/connectivity,device_info/device_info,espresso,google_maps_flutter/google_maps_flutter,google_sign_in/google_sign_in,image_picker/image_picker,in_app_purchase,integration_test,ios_platform_images,local_auth,package_info,path_provider/path_provider,quick_actions,sensors,shared_preferences/shared_preferences,url_launcher/url_launcher,video_player/video_player,webview_flutter,wifi_info_flutter/wifi_info_flutter,share"
+      script:
+        - ./script/incremental_build.sh build-examples --ipa
+        - ./script/incremental_build.sh xctest --target RunnerUITests --skip=$plugins_to_skip


theoretically we could save on resource time by building once per example and then running both drive-examples and xctest?

jmagman · 2020-10-22T18:01:29Z

.cirrus.yml

+      env:
+        plugins_to_skip: "battery/battery,camera,connectivity/connectivity,device_info/device_info,espresso,google_maps_flutter/google_maps_flutter,google_sign_in/google_sign_in,image_picker/image_picker,in_app_purchase,integration_test,ios_platform_images,local_auth,package_info,path_provider/path_provider,quick_actions,sensors,shared_preferences/shared_preferences,url_launcher/url_launcher,video_player/video_player,webview_flutter,wifi_info_flutter/wifi_info_flutter,share"
+      script:
+        - ./script/incremental_build.sh build-examples --ipa


Did you mean to include build-examples?

jmagman · 2020-10-22T18:01:33Z

.cirrus.yml

+        plugins_to_skip: "battery/battery,camera,connectivity/connectivity,device_info/device_info,espresso,google_maps_flutter/google_maps_flutter,google_sign_in/google_sign_in,image_picker/image_picker,in_app_purchase,integration_test,ios_platform_images,local_auth,package_info,path_provider/path_provider,quick_actions,sensors,shared_preferences/shared_preferences,url_launcher/url_launcher,video_player/video_player,webview_flutter,wifi_info_flutter/wifi_info_flutter,share"
+      script:
+        - ./script/incremental_build.sh build-examples --ipa
+        - ./script/incremental_build.sh xctest --target RunnerUITests --skip=$plugins_to_skip


How long does this take, does it need to be sharded? Should it be run against the flutter stable channel, too?

cyanglaz

@jmagman @amirh Updated to run xctest right after drive-example

jmagman

What's the expr: syntax error?
Might want to not print xcrun simctl list --json
Did any of the tests run? Might want to print for the plugins where it does pass.

./script/incremental_build.sh xctest --target RunnerUITests --skip=$plugins_to_skip_xctest
expr: syntax error
Checking for changed packages from 08706281d34277c28296df7f3a272e5e13e2ff35
No changes detected in packages.
No changes detected in packages.
Running for all packages
Looking for available simulators...
xcrun simctl list --json
{
  "devicetypes" : [
    {
      "name" : "iPhone 4s",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 4s.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-4s"
    },
    {
      "name" : "iPhone 5",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 5.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-5"
    },
    {
      "name" : "iPhone 5s",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 5s.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-5s"
    },
    {
      "name" : "iPhone 6 Plus",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 6 Plus.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-6-Plus"
    },
    {
      "name" : "iPhone 6",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 6.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-6"
    },
    {
      "name" : "iPhone 6s",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 6s.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-6s"
    },
    {
      "name" : "iPhone 6s Plus",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 6s Plus.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-6s-Plus"
    },
    {
      "name" : "iPhone SE",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone SE.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-SE"
    },
    {
      "name" : "iPhone 7",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 7.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7"
    },
    {
      "name" : "iPhone 7 Plus",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 7 Plus.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-7-Plus"
    },
    {
      "name" : "iPhone 8",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 8.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-8"
    },
    {
      "name" : "iPhone 8 Plus",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 8 Plus.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-8-Plus"
    },
    {
      "name" : "iPhone X",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone X.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-X"
    },
    {
      "name" : "iPhone Xs",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone Xs.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-XS"
    },
    {
      "name" : "iPhone Xs Max",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone Xs Max.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-XS-Max"
    },
    {
      "name" : "iPhone Xʀ",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone Xʀ.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-XR"
    },
    {
      "name" : "iPhone 11",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 11.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-11"
    },
    {
      "name" : "iPhone 11 Pro",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 11 Pro.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-11-Pro"
    },
    {
      "name" : "iPhone 11 Pro Max",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPhone 11 Pro Max.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPhone-11-Pro-Max"
    },
    {
      "name" : "iPad 2",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad 2.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-2"
    },
    {
      "name" : "iPad Retina",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Retina.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Retina"
    },
    {
      "name" : "iPad Air",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Air.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Air"
    },
    {
      "name" : "iPad mini 2",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad mini 2.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-mini-2"
    },
    {
      "name" : "iPad mini 3",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad mini 3.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-mini-3"
    },
    {
      "name" : "iPad mini 4",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad mini 4.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-mini-4"
    },
    {
      "name" : "iPad Air 2",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Air 2.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Air-2"
    },
    {
      "name" : "iPad Pro (9.7-inch)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Pro (9.7-inch).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Pro--9-7-inch-"
    },
    {
      "name" : "iPad Pro (12.9-inch)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Pro (12.9-inch).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Pro"
    },
    {
      "name" : "iPad (5th generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad (5th generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad--5th-generation-"
    },
    {
      "name" : "iPad Pro (12.9-inch) (2nd generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Pro (12.9-inch) (2nd generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Pro--12-9-inch---2nd-generation-"
    },
    {
      "name" : "iPad Pro (10.5-inch)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Pro (10.5-inch).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Pro--10-5-inch-"
    },
    {
      "name" : "iPad (6th generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad (6th generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad--6th-generation-"
    },
    {
      "name" : "iPad (7th generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad (7th generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad--7th-generation-"
    },
    {
      "name" : "iPad Pro (11-inch)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Pro (11-inch).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Pro--11-inch-"
    },
    {
      "name" : "iPad Pro (12.9-inch) (3rd generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Pro (12.9-inch) (3rd generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Pro--12-9-inch---3rd-generation-"
    },
    {
      "name" : "iPad mini (5th generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad mini (5th generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-mini--5th-generation-"
    },
    {
      "name" : "iPad Air (3rd generation)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/iPad Air (3rd generation).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.iPad-Air--3rd-generation-"
    },
    {
      "name" : "Apple TV",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/AppleTVOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple TV.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-TV-1080p"
    },
    {
      "name" : "Apple TV 4K",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/AppleTVOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple TV 4K.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-TV-4K-4K"
    },
    {
      "name" : "Apple TV 4K (at 1080p)",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/AppleTVOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple TV 4K (at 1080p).simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-TV-4K-1080p"
    },
    {
      "name" : "Apple Watch - 38mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch - 38mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-38mm"
    },
    {
      "name" : "Apple Watch - 42mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch - 42mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-42mm"
    },
    {
      "name" : "Apple Watch Series 2 - 38mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 2 - 38mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-2-38mm"
    },
    {
      "name" : "Apple Watch Series 2 - 42mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 2 - 42mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-2-42mm"
    },
    {
      "name" : "Apple Watch Series 3 - 38mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 3 - 38mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-3-38mm"
    },
    {
      "name" : "Apple Watch Series 3 - 42mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 3 - 42mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-3-42mm"
    },
    {
      "name" : "Apple Watch Series 4 - 40mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 4 - 40mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-4-40mm"
    },
    {
      "name" : "Apple Watch Series 4 - 44mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 4 - 44mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-4-44mm"
    },
    {
      "name" : "Apple Watch Series 5 - 40mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 5 - 40mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-5-40mm"
    },
    {
      "name" : "Apple Watch Series 5 - 44mm",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/DeviceTypes\/Apple Watch Series 5 - 44mm.simdevicetype",
      "identifier" : "com.apple.CoreSimulator.SimDeviceType.Apple-Watch-Series-5-44mm"
    }
  ],
  "runtimes" : [
    {
      "version" : "13.3",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/iPhoneOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/Runtimes\/iOS.simruntime",
      "isAvailable" : true,
      "name" : "iOS 13.3",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.iOS-13-3",
      "buildversion" : "17C45"
    },
    {
      "version" : "13.3",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/AppleTVOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/Runtimes\/tvOS.simruntime",
      "isAvailable" : true,
      "name" : "tvOS 13.3",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.tvOS-13-3",
      "buildversion" : "17K446"
    },
    {
      "version" : "6.1.1",
      "bundlePath" : "\/Applications\/Xcode-11.3.1.app\/Contents\/Developer\/Platforms\/WatchOS.platform\/Library\/Developer\/CoreSimulator\/Profiles\/Runtimes\/watchOS.simruntime",
      "isAvailable" : true,
      "name" : "watchOS 6.1",
      "identifier" : "com.apple.CoreSimulator.SimRuntime.watchOS-6-1",
      "buildversion" : "17S445"
    }
  ],
  "devices" : {
    "com.apple.CoreSimulator.SimRuntime.watchOS-6-1" : [
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple Watch Series 4 - 40mm",
        "udid" : "8C009CAC-EEDD-4C87-BE7A-AF45BE7975EF"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple Watch Series 4 - 44mm",
        "udid" : "FC34751E-B114-46A6-ACC6-9506B39FD32E"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple Watch Series 5 - 40mm",
        "udid" : "311D4E33-74E4-456A-B454-0842956E2037"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple Watch Series 5 - 44mm",
        "udid" : "42C5841A-9D75-4D64-9A9F-8339FCFF1D4C"
      }
    ],
    "com.apple.CoreSimulator.SimRuntime.tvOS-13-3" : [
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple TV",
        "udid" : "54C3B630-0279-4079-92E7-98E44029616A"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple TV 4K",
        "udid" : "8B07F84D-15B5-4528-A667-A321093B507E"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "Apple TV 4K (at 1080p)",
        "udid" : "23E1B808-588A-4789-8AC1-42C4126307DE"
      }
    ],
    "com.apple.CoreSimulator.SimRuntime.iOS-13-3" : [
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 8",
        "udid" : "BC9803C0-38F8-4DF7-A5DC-59414F5F8DA3"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 8 Plus",
        "udid" : "3B566CB8-E70B-4F37-9B17-6EF62042BE57"
      },
      {
        "state" : "Booted",
        "isAvailable" : true,
        "name" : "Flutter-iPhone",
        "udid" : "9071413D-675A-4C4A-9F5E-B1B74C2C1C49"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 11",
        "udid" : "4457A6A8-8B1B-4F3E-8088-5C11ACC6D539"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 11 Pro",
        "udid" : "018D0460-8ACD-4A17-8D7E-FD032CE17548"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPhone 11 Pro Max",
        "udid" : "9BD1C1A0-D2CF-49E9-A5A8-8917B123E4F4"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPad Pro (9.7-inch)",
        "udid" : "99D99D09-F54E-4936-B704-5A41E61BD13D"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPad (7th generation)",
        "udid" : "D29624A7-6ABB-43A1-A7F8-01CBD9A850F2"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPad Pro (11-inch)",
        "udid" : "091C64C9-863F-4AE8-A37C-4126F2B93630"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPad Pro (12.9-inch) (3rd generation)",
        "udid" : "2593F135-6DFE-41D7-BD4E-50C0BA7179F4"
      },
      {
        "state" : "Shutdown",
        "isAvailable" : true,
        "name" : "iPad Air (3rd generation)",
        "udid" : "124E9F0C-3AD2-4753-A790-94C9AF989A45"
      }
    ]
  },
  "pairs" : {
    "F50018E0-BEDF-4FD6-8CA3-11A2EAC9F56B" : {
      "watch" : {
        "name" : "Apple Watch Series 5 - 44mm",
        "udid" : "42C5841A-9D75-4D64-9A9F-8339FCFF1D4C",
        "state" : "Shutdown"
      },
      "phone" : {
        "name" : "iPhone 11 Pro Max",
        "udid" : "9BD1C1A0-D2CF-49E9-A5A8-8917B123E4F4",
        "state" : "Shutdown"
      },
      "state" : "(active, disconnected)"
    },
    "93669542-EBDB-4114-BA19-6377F3AF9ABE" : {
      "watch" : {
        "name" : "Apple Watch Series 5 - 40mm",
        "udid" : "311D4E33-74E4-456A-B454-0842956E2037",
        "state" : "Shutdown"
      },
      "phone" : {
        "name" : "iPhone 11 Pro",
        "udid" : "018D0460-8ACD-4A17-8D7E-FD032CE17548",
        "state" : "Shutdown"
      },
      "state" : "(active, disconnected)"
    }
  }
}
device selected: 
{state: Shutdown, isAvailable: true, name: iPad Air (3rd generation), udid: 124E9F0C-3AD2-4753-A790-94C9AF989A45}
Start running for url_launcher/url_launcher ...
url_launcher/url_launcher was skipped with the --skip flag.
Start running for url_launcher/url_launcher_linux ...
iOS is not supported by this plugin.
Start running for url_launcher/url_launcher_macos ...
iOS is not supported by this plugin.
Start running for url_launcher/url_launcher_platform_interface ...
iOS is not supported by this plugin.
Start running for url_launcher/url_launcher_windows ...
iOS is not supported by this plugin.
Start running for video_player/video_player ...
video_player/video_player was skipped with the --skip flag.
Start running for video_player/video_player_platform_interface ...
iOS is not supported by this plugin.
Start running for webview_flutter ...
webview_flutter was skipped with the --skip flag.
Start running for wifi_info_flutter/wifi_info_flutter ...
wifi_info_flutter/wifi_info_flutter was skipped with the --skip flag.
Start running for wifi_info_flutter/wifi_info_flutter_platform_interface ...
iOS is not supported by this plugin.
All XCTests have passed!

cyanglaz · 2020-10-22T20:16:13Z

What's the expr: syntax error?

I think the env var for skip plugins were not parsed correctly. I'm going to fix it.

Might want to not print xcrun simctl list --json

Makes sense, will update it in the flutter/plugin_tools

Did any of the tests run? Might want to print for the plugins where it does pass.

This PR won't run any tests as I intend to skip all plugins, but will add a print to print the passed plugins.

jmagman · 2020-10-22T21:36:36Z

Did any of the tests run? Might want to print for the plugins where it does pass.

This PR won't run any tests as I intend to skip all plugins, but will add a print to print the passed plugins.

Sorry, I now see your comment

As it for now, it skips all the plugins.

I think the env var for skip plugins were not parsed correctly. I'm going to fix it.

Once that's done I'll review! The other changes are for plugin_tools as you said. Hard to tell what needs changing until you actually use it! 😄

cyanglaz · 2020-10-22T22:03:18Z

@jmagman Yep, Ill make a PR there and ping you.

cyanglaz · 2020-10-23T22:14:33Z

@jmagman The output message looks good now. PTAL.

jmagman · 2020-10-23T23:32:48Z

It still says ./script/incremental_build.sh xctest --target RunnerUITests --skip $PLUGINS_TO_SKIP_XCTESTS expr: syntax error
but it does seem to be skipping them?

shared_preferences/shared_preferences was skipped with the --skip flag.

jmagman

LGTM, we can iterate more on the output when you turn the first one on.

cyanglaz · 2020-10-24T01:23:09Z

The expr: syntax error seems unrelated to this particular command. I saw the same error other incremental_build.sh too. I guess it is related to incremental_build.sh

cyanglaz · 2020-10-24T01:30:26Z

@jmagman Opened an issue for the error. I might cycle back to take a look at that some day
flutter/flutter#68929

…n the repo (#3210) * enable xctest in .cirrus.yml (#3189) * [share] Fix bug on iPad where `origin` is null and enable XCUITests (#3188)

…n the repo (flutter#3210) * enable xctest in .cirrus.yml (flutter#3189) * [share] Fix bug on iPad where `origin` is null and enable XCUITests (flutter#3188)

enable .cirrus.yml

17b889c

google-cla bot added the cla: yes label Oct 22, 2020

cyanglaz requested review from jmagman and amirh October 22, 2020 17:58

amirh reviewed Oct 22, 2020

View reviewed changes

jmagman reviewed Oct 22, 2020

View reviewed changes

run xtest with drive-examples

391af6e

cyanglaz commented Oct 22, 2020

View reviewed changes

cyanglaz force-pushed the enable_xctest branch from a7c4789 to 391af6e Compare October 22, 2020 19:10

jmagman reviewed Oct 22, 2020

View reviewed changes

env var

c488d8b

cyanglaz added 4 commits October 23, 2020 10:01

test skip

d34a1f6

simplify to test

f187c68

test

2bf9d1f

remove env var

52b2117

cyanglaz changed the title ~~enable .cirrus.yml~~ enable xctest in .cirrus.yml Oct 23, 2020

cyanglaz added 2 commits October 23, 2020 11:03

test

2af1248

revert

70b5d77

cyanglaz requested review from jmagman and amirh October 23, 2020 22:13

jmagman approved these changes Oct 23, 2020

View reviewed changes

cyanglaz merged commit 19517de into flutter:xctest Oct 26, 2020

cyanglaz deleted the enable_xctest branch October 26, 2020 16:02

cyanglaz pushed a commit that referenced this pull request Oct 27, 2020

[share] Fix bug on iPad where origin is null and enable XCUITests i…

642290b

…n the repo (#3210) * enable xctest in .cirrus.yml (#3189) * [share] Fix bug on iPad where `origin` is null and enable XCUITests (#3188)

cyanglaz mentioned this pull request Oct 27, 2020

[share] Fix bug on iPad where origin is null and enable XCUITests in the repo #3210

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enable xctest in .cirrus.yml #3189

enable xctest in .cirrus.yml #3189

cyanglaz commented Oct 22, 2020 •

edited

amirh Oct 22, 2020

jmagman Oct 22, 2020

jmagman Oct 22, 2020

cyanglaz left a comment

jmagman left a comment

cyanglaz commented Oct 22, 2020

jmagman commented Oct 22, 2020

cyanglaz commented Oct 22, 2020

cyanglaz commented Oct 23, 2020

jmagman commented Oct 23, 2020

jmagman left a comment

cyanglaz commented Oct 24, 2020

cyanglaz commented Oct 24, 2020

enable xctest in .cirrus.yml #3189

enable xctest in .cirrus.yml #3189

Conversation

cyanglaz commented Oct 22, 2020 • edited

amirh Oct 22, 2020

Choose a reason for hiding this comment

jmagman Oct 22, 2020

Choose a reason for hiding this comment

jmagman Oct 22, 2020

Choose a reason for hiding this comment

cyanglaz left a comment

Choose a reason for hiding this comment

jmagman left a comment

Choose a reason for hiding this comment

cyanglaz commented Oct 22, 2020

jmagman commented Oct 22, 2020

cyanglaz commented Oct 22, 2020

cyanglaz commented Oct 23, 2020

jmagman commented Oct 23, 2020

jmagman left a comment

Choose a reason for hiding this comment

cyanglaz commented Oct 24, 2020

cyanglaz commented Oct 24, 2020

cyanglaz commented Oct 22, 2020 •

edited