-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime, x/mobile: Severe performance regression since 1.18+ on iOS, when combined with Xcode #57651
Comments
cc @hyangah |
This is interesting. It would be good to know what is the difference Xcode and Xcode profiler cause.
The frame pointer has been enabled on iOS before Go 1.17. |
Oops, I should've cc'ed @golang/ios . |
Difficult to say for sure. But one thing we were able to say for sure is that when running XCode Profiler, the console/terminal output is not available. So perhaps that is a useful pointer. Another thing we notice is that when connecting the iOS device to XCode over Wifi, the issue is significantly worse than when connecting over USB (although even USB mode is significantly worse than not connecting to XCode at all)
Sorry, I seem to have made a mistake when creating the issue. The issue starts ocurring from Go 1.18 onwards, not 1.17 onwards. I have just updated the ticket title and and description accordingly |
Thanks. For 1.18, could you try building the code with GOEXPERIMENT=noregabi, to see if it is the register calling convention? Thanks. (Note that the GOEXPERIMENT only affects 1.18, not later versions of Go.) |
@cherrymui we just tried by making a Go 1.18.10 build with |
Which 1.17 release works OK? One potential culprit (if the XCode is doing something address-space proportional or... something) is https://go.dev/cl/344401. That CL was backported to 1.17 and 1.16 though, so it depends on which minor release you're using. I suppose you could try rolling that CL back in 1.18 and see what happens? |
Thanks for checking @mknyszek, but I'm afraid that 1.16.15 and 1.17.13 (both of which contain the CL in question) are working fine. Given that we are the original reporters of #46860 that lead to https://go.dev/cl/344401, we have been running 1.16.15 in production for quite some time now without any problems. |
Haha, no worries, I didn't expect you too to notice, just a funny coincidence :) Anyway, we'll get started on the bisect and report back the results. |
@mknyszek some good news. My colleague @mees-vdb (🥇) spent the day doing a bisect and we seem to have found the culprit. Fortunately it seems to essentially be a single-line: a4b2c57 In all honesty, I have no idea how this commit can have such a severe performance impact, but it seems it does. Perhaps @ianlancetaylor can shed some light on this? |
Thanks. Interesting. If this is only for running when connected to XCode, I think you can set the environment variable GODEBUG=asyncpreemptoff=1 to disable async preemption. In the past I remember LLDB had hard time handling signals (seems better now?). Maybe that is related. |
I can confirm that setting I don't understand enough of the underlying problem that a4b2c57 is solving to understand if there could be other ways of solving this though, ways that would not result in the issues with XCode without having to set GODEBUG. |
That change made sure the signal handler for asynchronous preemption signal is installed in c-archive and c-shared modes, so asynchronous preemption is enabled in those modes. Previously it was not enabled, due to neglect, not intentional. This changed fixed the issue. (iOS Go programs almost always use those build modes.) I think the slowdown is due to the slowness of signal handling in XCode/LLDB. As this only affect running under XCode (i.e. debugging), I think setting GODEBUG is actually the best option. Which version of XCode are you using? |
We tested on a various XCode versions, the oldest one being ~1.5 years old, and the newest one being the latest 14.2.
I'm a bit out of my depth here, but if the problem is Xcode/LLDB-related, isn't it odd that only threads created by the Go runtime suffer from it? |
The Go runtime just happens to send OS signals much more often than other programs due to the design of asynchronous preemption (what |
Anyone know if this issue affects Android? |
Given that it is specific to Xcode and LLDB, I don't think it affects Android (at least not in the same way). |
We (colleague of @rayvbr) have not observed this performance issue on Android. Indeed, this appears to be iOS-only. |
What version of Go are you using (
go version
)?1.18 and higher
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?ios/arm64
What did you do?
We’ve been noticing for a while now that starting with Go 1.18, performance for iOS apps that contain libraries written in Go has severely regressed, to the point where such apps have become unusable under certain conditions. Under those conditions, an otherwise empty app that is calling an .xcframework generated from Go code using gomobile/bind, is experiencing severe fps drops (going as low as 5fps) even though CPU load is close to 0.
Interestingly, the problem:
See attached for a very simple Go project, which is then compiled for iOS with gomobile/bind and wrapped in a Swift wrapper project. For convenience and ease of reproducibility, we’ve including both the full Swift Xcode project as well as pre-compiled .xcframework files based on Go 1.16.15 and 1.18.9 (in addition to the full Go source code). The actual Go project does nothing more than simulate some load by calculating a few sha265 hashes and then sleeping.
To reproduce: simply run the project in XCode, and observe the fps counter.
Note that the drop in performance doesn't just affect the Go code, but the full app containing the library, even code that does not depend in any way on the code being executed by the Go runtime.
Perhaps this is in some way connected to the stack frame pointers/new register-based calling convention introduced for ios/arm64 in Go 1.18?
What did you expect to see?
What did you see instead?
Performance being fine with Go 1.17 and lower, while being very bad on Go 1.18 and higher. While CPU load is close to 0.
Project to reproduce
go-ios-performance-regression-project.zip
The text was updated successfully, but these errors were encountered: