Permanent timeout in test-android-startup job

[jdm]

+ ./mach test-android-startup --release
Assuming --target i686-linux-android
Couldn't statvfs() path: No such file or directory
emulator: Requested console port 5580: Inferring adb port 5581.
emulator: WARNING: cannot read adb public key file: /root/.android/adbkey.pub
emulator: WARNING: Your AVD has been configured with an in-guest renderer, but the system image does not support guest rendering.Falling back to 'swiftshader_indirect' mode.
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.abm [bit 5]
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.abm [bit 5]
pulseaudio: pa_context_connect() failed
pulseaudio: Reason: Connection refused
pulseaudio: Failed to initialize PA contextaudio: Could not init `pa' audio driver
### WARNING: could not find /usr/share/zoneinfo/ directory. unable to determine host timezone
emulator: Cold boot: requested by the user
* daemon not running; starting now at tcp:5037
* daemon started successfully
### WARNING: could not find /usr/share/zoneinfo/ directory. unable to determine host timezone
emulator: INFO: boot completed
### WARNING: could not find /usr/share/zoneinfo/ directory. unable to determine host timezone
Success
Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=org.mozilla.servo/.MainActivity (has extras) }
--------- beginning of system
--------- beginning of main
--------- beginning of crash

[taskcluster:error] Task timeout after 1800 seconds. Force killing container.

https://tools.taskcluster.net/groups/X33ARXIMS5WDXIOk0PKwHg/tasks/LjWoG6vwQwSkWVwokP9dvA/runs/0/logs/public%2Flogs%2Flive.log

cc @SimonSapin
I have seen this on two jobs today so far.

[jdm]

This appears to be affecting every PR we try to merge.

[jdm]

Same problem on buildbot too.

[jdm]

--------- beginning of crash
Initialized EGL, version 1.4
Swap behavior 1
eglCreateContext: 0xea32f020: maj 2 min 0 rcv 2
eglMakeCurrent: 0xea32f020: ver 2 0 (tinfo 0xe9ce8a30)
HostConnection::get() New Host Connection established 0xebc08140, tid 3827
ro.sf.lcd_density must be defined as a build property
--------- beginning of crash
FATAL EXCEPTION: GLThread 217
Process: org.mozilla.servo, PID: 3808
java.lang.IllegalArgumentException: eglChooseConfig failed
	at android.opengl.GLSurfaceView$BaseConfigChooser.chooseConfig(GLSurfaceView.java:882)
	at android.opengl.GLSurfaceView$EglHelper.start(GLSurfaceView.java:1053)
	at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1429)
	at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1270)

[jdm]

I can reproduce this locally on mac with an old version of the emulator. I don't understand why this appeared suddenly.

[jdm]

When I run in the non-headless emulator and look at logcat, I see this:

11-14 09:26:19.844  4508  4525 I simpleservo: init
11-14 09:26:19.845  4508  4525 I simpleservo: Loading EGL...
11-14 09:26:19.851  4508  4525 I simpleservo: EGL loaded
11-14 09:26:19.851  4508  4525 I simpleservo: ResourceReader::read(prefs.json)
11-14 09:26:19.853  4508  4525 D simpleservo: WindowMethods::prepare_for_composite
11-14 09:26:19.853  4508  4525 D simpleservo: make_current
11-14 09:26:19.853  4508  4525 D simpleservo: WindowMethods::create_event_loop_waker
11-14 09:26:19.854  4508  4525 D simpleservo: WindowMethods::create_event_loop_waker
11-14 09:26:19.854  4508  4525 D simpleservo: WindowMethods::gl
11-14 09:26:19.856  4508  4532 D simpleservo: thread '<unnamed>' panicked at 'assertion failed: `(left != right)`
11-14 09:26:19.856  4508  4532 D simpleservo:   left: `Gles`,
11-14 09:26:19.856  4508  4532 D simpleservo:  right: `Gles`: gles must have compatible internal and external formats', /Users/jdm/.cargo/git/checkouts/webrender-c3596abe1cf4f320/790b76f/webrender/src/device/gl.rs:982:13
11-14 09:26:19.856  4508  4532 D simpleservo: note: Run with `RUST_BACKTRACE=1` for a backtrace.
11-14 09:26:19.867  4508  4525 E Gecko   : mozalloc_abort: Redirecting call to abort() to mozalloc_abort

[jdm]

This comes from https://github.com/servo/webrender/blob/02387f7e5c77c415cfa232366c321d6f621cae28/webrender/src/device/gl.rs#L1006-L1011. I think it's a separate problem from the headless rendering one, though.

[jdm]

More specifically - -gpu host triggers the webrender assertion. -gpu guest triggers:

11-14 09:54:57.270  4088  4106 E AndroidRuntime: FATAL EXCEPTION: GLThread 236
11-14 09:54:57.270  4088  4106 E AndroidRuntime: Process: org.mozilla.servo, PID: 4088
11-14 09:54:57.270  4088  4106 E AndroidRuntime: java.lang.IllegalArgumentException: No configs match configSpec
11-14 09:54:57.270  4088  4106 E AndroidRuntime: 	at android.opengl.GLSurfaceView$BaseConfigChooser.chooseConfig(GLSurfaceView.java:888)
11-14 09:54:57.270  4088  4106 E AndroidRuntime: 	at android.opengl.GLSurfaceView$EglHelper.start(GLSurfaceView.java:1053)
11-14 09:54:57.270  4088  4106 E AndroidRuntime: 	at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1429)
11-14 09:54:57.270  4088  4106 E AndroidRuntime: 	at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1270)
11-14 09:54:57.272  1659  1670 W ActivityManager:   Force finishing activity org.mozilla.servo/.MainActivity

[jdm]

I also see this in logcat:

11-14 09:55:03.094  4149  4149 D libEGL  : Emulator has vendor provided software renderer, qemu.gles is set to 2.
11-14 09:55:03.095  4149  4149 E vndksupport: Could not load /vendor/lib/egl/libGLES_swiftshader.so from sphal namespace: dlopen failed: library "/vendor/lib/egl/libGLES_swiftshader.so" not found.
11-14 09:55:03.095  4149  4149 E libEGL  : load_driver(/vendor/lib/egl/libGLES_swiftshader.so): unknown
11-14 09:55:03.096  4149  4149 D libEGL  : loaded /vendor/lib/egl/libEGL_swiftshader.so
11-14 09:55:03.104  4149  4149 D libEGL  : loaded /vendor/lib/egl/libGLESv1_CM_swiftshader.so
11-14 09:55:03.110  4149  4149 D libEGL  : loaded /vendor/lib/egl/libGLESv2_swiftshader.so