Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
x/build: sharded mobile builders #23824
Currently, the android/amd64 and android/386 builders run on an Android emulator with a amd64 system image. This is taxing the heavyly loaded mobile builder Mac Mini. It is also inefficient: the emulator builds competes with Android device builds and any concurrent iOS builds. Because builds are slow, android is not in the trybot set, leaving me to often pester CL authors with "this CL broke android".
Is it possible to add (and run) the Android emulator inside the existing docker images used for regular builders and then run android sharded tests as any other GOOS? If so, android/amd64 and android/386 builds would complete much faster, and take some pressure off the mobile device builder.
If the emulator builds are stable enough, android/amd64 (and perhaps android/386) could even join the trybot set, avoiding most (if not all) android specific followup work after a CL is submitted.
seems to indicate that the standard emulator supports headless mode.
Sure, prepare a Docker image that lets us do the traditional make.bash, then snapshot, then clone to several mirrors, then run tests one-at-a-time.
It'll require some tweaking of the build system probably, but I'll do those parts if you do the bulk of the work in preparing the Dockerfile.
headless helps but isn't required. Worst case we could run a headless X server so it thinks it has a window.
The following Dockerfile
And supporting run-emulator.sh (as well as a Go checkout in
Can be started with:
Where I'm able to complete androidtest.sh with android/386 and android/amd64:
I can even complete a full test of gomobile:
Let me know how to proceed.
Sorry, I realistically won't be able to get to start on this until April 1st. I'm preparing for an upcoming Gophercon talk this weekend, traveling next week for the talk, and then on vacation the last week of March.
I might be able to work on this next week from Moscow, but no promises.
@andybons has been wanting to learn how this whole system works more, so this might be a good "starter project" for adding a builder.
BTW, I forgot to ask earlier: how long does this all take? I imagine sharding is necessary for being a trybot, but is your goal for it to be a trybot right away, or just to have it run on GCE by us in Kubernetes? If it's easier, we could skip sharding for the first version.
But I'm also fine with sharding for the first version, but curious about timing. I imagine make.bash is still fast like always (1 minute or less), but then how long are the various tests?
No problem at all. My ping was just as much to see if the Docker image was enough.
My wishlist ordered by importance:
(And in the future: 4. Do 1-3 for a darwin builder running an 386/amd64 iOS emulator.)
In other words, "just" a regular non-sharded builder on GCE would make me very grateful.
On my i7-6700K@4GHz with 32GB memory with a running Android 8 x86_64 emulator, ./androidtest.bash takes ~2min and ./all.bash on the same machine takes ~3min.
BTW, I'm not even sure trybots can be stable enough before #23795 is fixed. You wouldn't happen to know someone to ping for the corresponding adb bug, https://issuetracker.google.com/issues/73230216 ?