"Run app to dismiss dialogs" flakey with iOS 17 devices #141383
Comments
|
cc @jmagman |
|
|
So with iOS 17 devices, it automatically makes all paired devices also wirelessly connected. Did a bit of research and found this under Xcode 15 Known Issues:
So perhaps it's a bug with Xcode 15? I don't see any mention of it being resolved in newer Xcode releases, but we could try Xcode 15.2. |
|
@yusuf-goog Do you know if the devices are connected to the network? |
|
@vashworth they are connected to wifi. Do you want us to put them into airplane modE? |
|
@yusuf-goog Try creating a new vanilla Xcode project and deploy to these phones at least once. That should confirm they are set up for development. It wouldn't be related to Flutter or our particular CI setup. |
|
Will try deployment on one of the testbeds and get provide an update. |
Potential fix for flutter#141383.
Reverts flutter#141392 Initiated by: QuncCccccc This change reverts the following previous change: Original Description: Potential fix for flutter#141383.
|
We have opened the device with Xcode 15.2 on mac-39. Please confirm that the device is now in the expected state. Once thats done, we will setup all remaining iphones which are on iOS 17. |
The device on mac-39 still exhibits this issue: https://luci-milo.appspot.com/ui/p/flutter/builders/staging/Mac_arm64_ios%20flavors_test_ios_xcode_debug/1193/overview. I believe all of the devices are properly set up. All of them have passed the step and ran tests successfully. I tried upgrading to Xcode 15.2 to see if that resolves it, but my PR got reverted. #141420 should reland it. If the upgraded Xcode version doesn't fix it, then perhaps we try airplane mode. |
|
Actually, lets go ahead and put flutter-devicelab-mac-10's device in airplane mode. @yusuf-goog Let me know when it's done please |
|
@vashworth mac-10 device has been put into airplane mode. |
|
Confirmed it's not fixed by using Xcode 15.2: https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_arm64_ios%20flavors_test_ios_xcode_debug/1198/overview |
|
Maybe the cmd line --destination id arg has changed with xcode 15? Xcode can connect fine to the device when using the xcode app on mac-39 |
|
On mac-39, the device showed up correctly under Xcode "devices and simulators" and I was able to click the "take screenshot" button without any warning. But it saved screenshots as black screens. when trying to deploy a test app on the device, received error message about signing, and had to go through the flow of logging into apple id and importing the provisioning profile into Xcode (https://screenshot.googleplex.com/6SDaGHypNLBggMP.png). After which, screenshots are now appearing as expected. Note, that before the above import, Xcode showed the Lets monitor the builds on mac-39 and see if this has some effect on the failures we are seeing: https://chromium-swarm.appspot.com/bot?id=flutter-devicelab-mac-39 |
|
Looks like airplane mode on mac-10 did not work. Also, manual setup for mac-39 also did not work. I think we should try moving this https://flutter.googlesource.com/recipes/+/refs/heads/main/recipe_modules/os_utils/api.py#369 to run before the step - hopefully it can give us some information, such as what connection type it's using |
Some VPN/ port issue on Ventura? https://developer.apple.com/forums/thread/741537 Strange it's happening intermittently. Maybe it's fixed on macOS 14. |
|
These devices are all connected directly to the host via cable, so the network/lan issues should not apply? |
That's what that issue and this one are reporting--they are tethered. |
Although they're connected via USB, iOS 17 uses a new device connectivity stack which I believe always uses the network in a way. Some sources of info: https://developer.apple.com/forums/thread/737875?answerId=765840022#765840022
https://github.com/doronz88/pymobiledevice3/blob/master/misc/RemoteXPC.md#usb-ethernet
|
|
@yusuf-goog Can you upgrade https://chromium-swarm.appspot.com/bot?id=flutter-devicelab-mac-5 to macOS 14 to see if that could fix? |
Share your connection from : Ethernet |
done |
|
@vashworth wifi/bluetooth has also been turned off on mac-10 and mac-26 iphones, and put into airplane mode. |
|
Thanks @yusuf-goog and @kentnguyen99! I'll continue to monitor and see if that fixes |
|
Issue continues to persist across all bots. |
|
Can we get a sysdiagnose off the Mac right after this reproduces? |
@jmagman Are you suggesting we try to automate it in recipes to run |
|
Also, I'm implementing another retry process that if it fails, try reseting and re-selecting Xcode. If it fails again, try clearing Xcode cache and re-installing: https://flutter-review.googlesource.com/c/recipes/+/54300 It seems to me that the only thing that happens between builds where one fails with this flake and one passes is that Xcode is reset and time has passed. And since waiting up to 10 minutes for it to connect doesn't seem to work, I thought we could try resetting Xcode. |
|
So I was able to catch this happening I did observe that Finder was still showing the device. Also, when I turned on QuickTime to view the device, it suddenly connected (which cause step 39.4.5 to pass). QuickTime showed that the device was booted and on. So seems like the device is still connected, but the new technology Xcode uses to check if the device is connected doesn't work sometimes. Also, beforeI ran the CL, I was VNC'd to the bot and I got kicked off. I had to relogin. My ssh terminal had this error: Maybe the host bot restarted and that caused the device disconnect? |
Manually once to see in the logs if anything weird is going on. |
|
Okay I went down a rabbit hole with sysdiagnose logs and discovered a pattern. It seems like occasionally our bots restart after tests. On startup, it does something with LaunchServices. For example: However, occasionally on startup, these LaunchServices for ^^ I analyzed 18 restarts (on 2 different bots) and found this to always be true. I'm not entirely sure what these processes are doing. Perhaps @jmagman Any insight into LaunchServices and how that works? |
|
I tried to make and sign a com.apple.launchservices logging profile to show the I see |
|
Since one of the times I was able to recreate this issue, it was fixed by opening QuickTime. I'm adding an automation to open QuickTime with the device to see if it will work as a temporary workaround: https://flutter-review.googlesource.com/c/recipes/+/54460 Seems like it works: https://chromium-swarm.appspot.com/task?id=675b1d295e24b310 |
|
The QuickTime solution appears to be working 🤯 I'll monitor for another day, but I think it fixed the issue |
Thanks for figuring out this extremely strange but effective workaround! |
|
@vashworth This is great! Lets have it bake for a bit and then we can begin the rollout again. Even if its effective, its a precarious path having these types of workarounds in place. Lets leave this, or another issue open to track rootcause/fix for this. |
|
I also filed an issue with Apple FB13559148 tracked internally at b/322225404 |
|
Workaround has successfully worked for a few days now! As @yusuf-goog requested, I'll leave this open but deprioritize |
vashworth
added
P2
Bug: flutter/flutter#141383, flutter/flutter#141624 Change-Id: I31209c422b3a00ecd0da60ac71ff07c2d2d6ce39 Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/54108 Reviewed-by: Keyong Han <keyonghan@google.com> Commit-Queue: Victoria Ashworth <vashworth@google.com>
Bug: flutter/flutter#141383 Change-Id: I7c8f223e460eb58e1ccf4f879903c1da8b574619 Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/54162 Reviewed-by: Yusuf Mohsinally <mohsinally@google.com> Commit-Queue: Victoria Ashworth <vashworth@google.com>
…to connect Bug: flutter/flutter#141383 Change-Id: Ib161ef48b36b8790df6fe84b1ba9124cb43ed4e2 Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/54460 Reviewed-by: Keyong Han <keyonghan@google.com> Reviewed-by: Yusuf Mohsinally <mohsinally@google.com> Commit-Queue: Victoria Ashworth <vashworth@google.com>
Type of Request
bug
Infrastructure Environment
Recipes
What is happening?
Seems that the "Run app to dismiss dialogs" step is flakey when using iOS 17 devices. In this step, we test the Xcode project
infra-dialog.xcodeprojusing the device id. For some reason,xcodebuilddoes not find the deviceDoes not to appear to be specific to a particular device, seems to be happening to all of the iOS 17 devices. Also does not happen consistently.
Here are a few examples:
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_arm64_ios%20flavors_test_ios_xcode_debug/1152/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_arm64_ios%20flavors_test_ios_xcode_debug/1162/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_arm64_ios%20flavors_test_ios_xcode_debug/1168/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20flutter_gallery_ios__start_up_xcode_debug/2409/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20integration_ui_ios_driver_xcode_debug/2413/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20integration_ui_ios_driver_xcode_debug/2416/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20integration_ui_ios_driver_xcode_debug/2420/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20microbenchmarks_ios_xcode_debug/1848/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20microbenchmarks_ios_xcode_debug/1853/overview
https://ci.chromium.org/ui/p/flutter/builders/staging/Mac_ios%20microbenchmarks_ios_xcode_debug/1876/overview
Steps to reproduce
No response
Expected results
No response
The text was updated successfully, but these errors were encountered: