Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS Simulator screenshot - invalid device error #9

Closed
zanuka opened this Issue Mar 4, 2019 · 26 comments

Comments

Projects
None yet
2 participants
@zanuka
Copy link

zanuka commented Mar 4, 2019

Hi Maurice, I am having some issues getting this to run on iOS Simulator

steps to reproduce:

  1. target iPhone 7 Plus in screenshots.yaml
  2. launch iOS simulator and with iPhone 7 Plus device opened
  3. setup basic driver test with await screenshot(driver, config, 'SplashScreen');
  4. run screenshots

currently getting the following error:

❯ screenshots
Clearing images in ios/fastlane/screenshots/en-AU for 'iPhone 7 Plus'...
Starting simulator 'iPhone 7 Plus' in locale en-AU ...
Invalid device: E96E4A16-72F8-4841-BF60-381F0BB424B0) (Shutdown
Unhandled exception:
command failed: cmd='xcrun simctl boot E96E4A16-72F8-4841-BF60-381F0BB424B0) (Shutdown'
#0      cmd (package:screenshots/utils.dart:36:5)
#1      simulator (package:screenshots/screenshots.dart:202:5)
#2      run (package:screenshots/screenshots.dart:66:9)
<asynchronous suspension>
#3      main (file:///Users/zanuka/.pub-cache/hosted/pub.dartlang.org/screenshots-0.1.4/bin/main.dart:63:9)
<asynchronous suspension>
#4      _startIsolate.<anonymous closure> (dart:isolate/runtime/libisolate_patch.dart:287:32)
#5      _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)

the same driver test is capturing screenshot fine on Android emulators and placing files in the right directory - android/fastlane/metadata/android/en-AU/

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

Yup, this is the second time I've run into this problem (#2).
It looks like the output from

xcrun simctl list

differs significantly from installation to installation of xCode.

I think I have a fix for it by parsing the output from the following instead:

xcrun simctl list devices --json

I will give it a shot and post pack here.

Thnx

BTW: Screenshots expects no emulators/simulators to be running at startup. I will try to add a check for that too.

@mmcc007 mmcc007 added the bug label Mar 4, 2019

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

Ok, thanks for looking into it and for the additional info

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

I tried running it again on my MacBook Air and get similar results:

  • driver tests / screenshots run fine on Android emulators
  • similar failures on iOS
  • iOS simulator doesn't actually spawn like the Android ones do

output from my test run that might be useful:

Processing screenshots from test...
Moving screenshots to android/fastlane/metadata/android/en-AU/images/phoneScreenshots
Stopping emulator: Nexus_6P ...
Starting simulator 'iPhone X' in locale en-AU ...
Capturing screenshots with test test_driver/smoke.dart on simulator iPhone X in locale en-AU ...
Using device iPhone X.
Starting application: test_driver/smoke.dart
Running pod install...                                             10.5s
Running Xcode build...                                          
Xcode build done.                                           53.3s
flutter: Observatory listening on http://127.0.0.1:51042/
00:00 +0: smoke test (setUpAll)
[Crashlytics] Version 3.12.0 (136)
00:01 +0: smoke test SplashScreen > verify signup button is rendered

also this was toward the bottom of output:

Driver tests failed: 255
Unhandled exception:
command failed: cmd='flutter drive test_driver/smoke.dart'
#0      streamCmd (package:screenshots/utils.dart:63:7)
<asynchronous suspension>
#1      screenshots (package:screenshots/screenshots.dart:100:9)
<asynchronous suspension>
#2      run (package:screenshots/screenshots.dart:70:17)
<asynchronous suspension>
#3      main (file:///Users/method/.pub-cache/hosted/pub.dartlang.org/screenshots-0.1.4/bin/main.dart:63:9)
<asynchronous suspension>
#4      _startIsolate.<anonymous closure> (dart:isolate/runtime/libisolate_patch.dart:287:32)
#5      _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)

I'm on XCode 10.1, MacOS 10.14.3

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

I'm able to parse the new output now... so will be adding some more emulator/simulator initial conditions checks and then manually test.

I have tested the current release on my MacBook Air and it ran fine. So, not being able to reproduce the problem locally, I'm coding blind (sort of).

There may be some regional differences between MacBook Air's shipped by Apple, who knows (no way currently for me to test). I expect the new code will help cover some of these differences (... at least the ones we currently know of).

When complete, I will test the latest code on a MacBook Air (more of a sanity test of existing features and not really a test of the reason for the new code), release, and go from there.

If it runs on your MacBook Air, I would then consider this bug fixed. 👍

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

BTW: I can post the pre-release code to the dev branch and co-ordinate testing on your MacBook Air before releasing, if that works for you.

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

Thanks for looking into this Maurice, just got back online...
It would be great if you could post up the pre-release code on a dev branch, I'd be happy to test ⭐️

I am also going to run the screenshots job in our Travis CI pipeline as per your example project config. Nice to hear it's working locally on your Air.

I have also tested in on an iMac (also on XCode 10.1) and the same errors occur on the iOS screenshot job run.

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

Great!

BTW: It's may not be worth trying to run Screenshots on Travis. It works for iOS simulators, but have not yet found a way to configure the android emulator to run graphics correctly. See the screenshots in artifacts at https://github.com/mmcc007/screenshots/releases
There may well be a way to configure the emulator correctly. If so, have not found it.

In the end may have to pitch to real devices via Travis (using upcoming integration with AWS Device Farm, which works for android)

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

Understood, perhaps just disregard issue #10 I just created for now while testing the Travis flow.

Would be awesome to get the iOS device screenshots working locally...

It's the last step in our deployment checklist, then we can submit to both stores 🎉

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

Even though it sounds like a roundabout process, if I could get the iOS screenshots to generate via Travis and do the Android ones locally, that would do the trick for now

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

Both should be working now in dev branch (according to manual testing on MacBook Air and auto-test on Travis https://travis-ci.com/mmcc007/screenshots/builds/103114803 --emulator config problem of course still exists on Travis--).

Clone the screenshots repo and checkout the dev branch:

git clone https://github.com/mmcc007/screenshots.git
git checkout dev

You will need to link Screenshots to your local dev using:

pub global activate --source path <path to screenshots repo>

Give it a shot and see if it fixes the problem on your local MacBook Air 🤞

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

Ok I just tried running from your updated dev branch and received the following output, even though I have no simulators/emulators currently running:

#######################################################################
# Screenshots expects no devices to be running at startup. 
# Please disconnect all devices and shutdown all simulators/emulators.
#######################################################################
@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

Run

flutter devices

it should be finding something running
(sometimes iOS simulators are running headless without the user knowing)

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

yeah my bad, it was my connected iPhone 🤦‍♂️

disconnected that, and it's now running correctly on my local Air 💨

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

however, the smoke test that passes on Android emulators is failing on ios...

ext.flutter.driver errors keep spewing on each test

like so:

06:33 +4 -13: smoke test LocalAuthScreen > take screenshot [E]
  DriverError: Failed to fulfill WaitUntilNoTransientCallbacks due to remote error
  Original error: Bad state: The client closed with pending request "ext.flutter.driver".
  Original stack trace:
  #0      new Client.withoutJson.<anonymous closure> (package:json_rpc_2/src/client.dart:70:24)
  #1      StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
  #2      StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:119:48)
  #3      _rootRun (dart:async/zone.dart:1120:38)
  #4      _CustomZone.run (dart:async/zone.dart:1021:19)
  #5      _FutureListener.handleWhenComplete (dart:async/future_impl.dart:150:18)
  #6      Future._propagateToListeners.handleWhenCompleteCallback (dart:async/future_impl.dart:609:39)
  #7      Future._propagateToListeners (dart:async/future_impl.dart:665:37)
  #8      Future._propagateToListeners (dart:async/future_impl.dart:566:9)
  #9      Future._completeWithValue (dart:async/future_impl.dart:483:5)
  #10     Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:513:7)
  #11     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
  #12     StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:119:48)
  #13     _rootRun (dart:async/zone.dart:1124:13)
  #14     _CustomZone.run (dart:async/zone.dart:1021:19)
  #15     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
  #16     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
  #17     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
  #18     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
  #19     _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:391:30)
  #20     _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
  #21     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)
@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 4, 2019

The Flutter Team has made some changes to how they handle timeouts. I tested on version 1.0.0 (stable channel). I will try on latest version. In the meantime, you might want to try on v1.0.0 using something like:

flutter version --force v1.0.0
@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

Ok thanks for the info on timeout, good to know

I figured out the tests were failing due to a different issue

I had the Simulator.app closed when running the tests initially. So I then tried opening the Simulator first, but with no hardware devices loaded, just the skeleton app. Then when running screenshots locally all my tests passed and screenshots were generated! WIN 💯

Interesting that locally you have to have the app open. Also trying this in our Travis pipeline...

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 4, 2019

just discovered that when I tried running the supported ios devices I get a new error:

Warning: multiple versions of 'iPhone X' detected.
  Using 'iPhone X' with iOS: iOS 12.1, ID: 1DF65DB2-6632-4AB7-88FA-58A897A54DAF.
Warning: multiple versions of 'iPhone 7 Plus' detected.
  Using 'iPhone 7 Plus' with iOS: iOS 12.1, ID: B6BA0092-0A54-480F-87D2-ED8524D85365.
Warning: multiple versions of 'iPad Pro (12.9-inch) (2nd generation)' detected.
  Using 'iPad Pro (12.9-inch) (2nd generation)' with iOS: iOS 12.1, ID: 7B0572DB-3891-4332-928D-3A8813B3BED6.
Clearing images in ios/fastlane/screenshots/en-AU for 'iPhone X'...
Clearing images in ios/fastlane/screenshots/en-AU for 'iPhone 7 Plus'...
Clearing images in ios/fastlane/screenshots/en-AU for 'iPad Pro (12.9-inch) (2nd generation)'...
simulatorInfo={availability: (available), state: Shutdown, isAvailable: true, name: iPhone X, udid: 1DF65DB2-6632-4AB7-88FA-58A897A54DAF, availabilityError: }
Starting simulator 'iPhone X' in locale en-AU ...
Capturing screenshots with test test_driver/smoke.dart on simulator iPhone X in locale en-AU ...
Using device iPhone X.
Starting application: test_driver/smoke.dart
Upgrading project.pbxproj of Runner.app' to include the 'flutter_assets' directory
Running pod install...                                           4.3s
Starting Xcode build...                                         
Xcode build done.                                           14.2s
Sending crash report to Google.

and this was after running screenshots with No devices detected.

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 5, 2019

Huh, interesting... I tried with no Simulator.app running and it ran fine.

...

The warnings are just to inform user that there are multiple simulators with the same name installed (which is not unusual for XCode)... and that Screenshots is using the first one it finds and continuing.

I can't tell what the

Sending crash report to Google.

is about. It may be unrelated to Screenshots.

Is there any more info on the crash? Is it reproducible?

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 5, 2019

Let me know if this bug is fixed and I will do a release.

Thanks for all the feedback.. it was very helpful.

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 5, 2019

yeah that might be due to our crashlytics implementation, not quite sure... doing some more testing now to see if I can reproduce on 2 different macs

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 5, 2019

Is this reproducible using Flutter Driver from the command line directly?

BTW: Can you create a new issue for this. I'd like to close this one out with the bug fix PR. 👍

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 5, 2019

cool, issue #13 created, this one can be closed.

yeah w/ Simulator app closed...

❯ flutter driver --target=test_driver/smoke.dart
No devices found.
@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 5, 2019

one thing I noticed (before closing) is that the dev branch changes seem to have broken the Android driver tests I'm running. If I go back to master version I can run them fine. are you seeing that?

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 5, 2019

I switched back to Flutter version 1.2.1 and Android driver tests run fine, so it must be related to the timeout changes you mentioned above

@mmcc007

This comment has been minimized.

Copy link
Owner

mmcc007 commented Mar 5, 2019

Interesting.... glad u got it working.

For me, running an (simple) integration test on android or iOS via Screenshots has always been consistent. Never ran into a problem. The problem has always been (so far) for Screenshots to set-up the correct conditions for the test to run.

@mmcc007 mmcc007 closed this in #12 Mar 5, 2019

@zanuka

This comment has been minimized.

Copy link
Author

zanuka commented Mar 5, 2019

Yes I think it is caused by the fact that my test is doing more complicated things, stepping through auth flows, etc... will have a play w/ the timeouts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.