PlayerSurface on API 34 distorts when prepare isn't manually called on a portrait video when using SURFACE_TYPE_SURFACE_VIEW

[eygraber]

Version

Media3 1.8.0

More version details

No response

Devices that reproduce the issue

Emulator running API 34

Devices that do not reproduce the issue

Emulator running API 30
Emulator running API 33
Emulator running API 35
Pixel 9 running API 36

Reproducible in the demo app?

Yes

Reproduction steps

Add the repro video to the first position in videos.kt, remove the call to prepare in initializePlayer, and change the call to MediaPlayerScreen to:

player?.let {
    Box(
      modifier = Modifier.fillMaxSize()
    ) {
      MediaPlayerScreen(player = it, modifier = modifier.fillMaxSize())
    }
  }

Expected result

The media plays correctly.

Actual result

The media starts playing back distorted, until the screen is interacted with.

repro.webm

Media

https://www.pexels.com/download/video/4058073/

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[marcbaechinger]

Did you see this happen on a real device or only on an emulator?

[eygraber]

I don't have an API 34 device to test with, so just the emulator.

[oceanjules]

Hi @eygraber,

A couple of points from me about Compose

• I think you are still using the old demo-compose from the release branch. Please try out the one on the main branch as well, it comes with a video choosing activity. Shouldn't make any difference for this bug, just a heads up about the ongoing development in the area, so that you don't get stuck behind.
• MediaScreenPlayer (or MainScreen as documented on the main branch) is a Box itself. There is no point in putting it into another Box unless you are grouping it with other elements.

And now for the actual problem:

• What are you trying to achieve by removing prepare()? That call is crucial for getting the resources like codecs, so you shouldn't be able to see anything at all (and definitely not an ongoing playback)
• The trick is - you are clicking on the PlayPauseButton which has the effect of calling the following method. Given that the player never left the IDLE state, we prepare and play. In the process of preparation, the video is decoded and we are able to extract things like size/aspect ratio/other metadata.
  

  media/libraries/common/src/main/java/androidx/media3/common/util/Util.java

  Lines 3806 to 3824 in bfe5930

  	public static boolean handlePlayButtonAction(@Nullable Player player) {
  	if (player == null) {
  	return false;
  	}
  	@Player.State int state = player.getPlaybackState();
  	boolean methodTriggered = false;
  	if (state == Player.STATE_IDLE && player.isCommandAvailable(COMMAND_PREPARE)) {
  	player.prepare();
  	methodTriggered = true;
  	} else if (state == Player.STATE_ENDED
  	&& player.isCommandAvailable(COMMAND_SEEK_TO_DEFAULT_POSITION)) {
  	player.seekToDefaultPosition();
  	methodTriggered = true;
  	}
  	if (player.isCommandAvailable(COMMAND_PLAY_PAUSE)) {
  	player.play();
  	methodTriggered = true;
  	}
  	return methodTriggered;

• My hypothesis is that the screen would be distorted even without you interacting with it. It's just the artifact of late preparation. If you could try using the demo from the main branch, you will see that we moved from PlayerSurface to ContentFrame. It's just a wrapper of PlayerSurface that takes care of resizing according to a predefined scale type (Fit, Crop, Zoom) AND allows you to specify the shutter (overlay that hides the fact that the video size is not available yet). You have to choose what the best user experience is for you - to see a surface that occupies max size while Compose is waiting for the Player to tell it the real decoded video size. Or you see a black frame for a second.

---

• As Marc noted, it's important to test on a real device - emulators often have limited hardware acceleration and may not fully support decoding very high-resolution videos like 4K (I think yours is 4096x2160). Even with a prepare call, I get a stacktrack like:

Click me to see the trace

Playback error
androidx.media3.exoplayer.ExoPlaybackException: MediaCodecVideoRenderer error, index=0, format=Format(1, null, video/mp4, video/avc, avc1.640034, 14042967, und, [2160, 4096, 25.0, ColorInfo(BT709, Limited range, SDR SMPTE 170M, false, 8bit Luma, 8bit Chroma)], [-1, -1]), format_supported=YES
at androidx.media3.exoplayer.BaseRenderer.createRendererException(BaseRenderer.java:558)
at androidx.media3.exoplayer.BaseRenderer.createRendererException(BaseRenderer.java:527)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.maybeInitCodecOrBypass(MediaCodecRenderer.java:598)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.onInputFormatChanged(MediaCodecRenderer.java:1700)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.onInputFormatChanged(MediaCodecVideoRenderer.java:1574)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.readSourceOmittingSampleData(MediaCodecRenderer.java:1124)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:901)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.render(MediaCodecVideoRenderer.java:1350)
at androidx.media3.exoplayer.RendererHolder.render(RendererHolder.java:392)
at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1375)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:704)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loopOnce(Looper.java:201)
at android.os.Looper.loop(Looper.java:288)
at android.os.HandlerThread.run(HandlerThread.java:67)

Caused by: androidx.media3.exoplayer.mediacodec.MediaCodecRenderer$DecoderInitializationException: Decoder init failed: c2.goldfish.h264.decoder, Format(1, null, video/mp4, video/avc, avc1.640034, 14042967, und, [2160, 4096, 25.0, ColorInfo(BT709, Limited range, SDR SMPTE 170M, false, 8bit Luma, 8bit Chroma)], [-1, -1])
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.maybeInitCodecWithFallback(MediaCodecRenderer.java:1229)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.maybeInitCodecOrBypass(MediaCodecRenderer.java:596)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.onInputFormatChanged(MediaCodecRenderer.java:1700) 
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.onInputFormatChanged(MediaCodecVideoRenderer.java:1574) 
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.readSourceOmittingSampleData(MediaCodecRenderer.java:1124) 
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:901) 
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.render(MediaCodecVideoRenderer.java:1350) 
at androidx.media3.exoplayer.RendererHolder.render(RendererHolder.java:392) 
at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1375) 
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:704) 
at android.os.Handler.dispatchMessage(Handler.java:102) 
at android.os.Looper.loopOnce(Looper.java:201) 
at android.os.Looper.loop(Looper.java:288) 
at android.os.HandlerThread.run(HandlerThread.java:67) 

Caused by: java.lang.IllegalArgumentException
at android.media.MediaCodec.native_configure(Native Method)
at android.media.MediaCodec.configure(MediaCodec.java:2215)
at android.media.MediaCodec.configure(MediaCodec.java:2131)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecAdapter.initialize(AsynchronousMediaCodecAdapter.java:194)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecAdapter.access$100(AsynchronousMediaCodecAdapter.java:53)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecAdapter$Factory.createAdapter(AsynchronousMediaCodecAdapter.java:134)
at androidx.media3.exoplayer.mediacodec.DefaultMediaCodecAdapterFactory.createAdapter(DefaultMediaCodecAdapterFactory.java:163)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.initCodec(MediaCodecRenderer.java:1308)

[eygraber]

There is no point in putting it into another Box unless you are grouping it with other elements.

In my actual use case an other Box is required for other elements.

What are you trying to achieve by removing prepare()

In my actual use case, we don't want to load the video until the user presses play because it is very large, we don't want to cache it, and we don't anticipate it being viewed very often

My hypothesis is that the screen would be distorted even without you interacting with it

I'm not sure I follow, because the screen is distorted without interacting with it. If you mean it would eventually get fixed even without interacting with it, that is not the case.

it's important to test on a real device

100%, but I don't have one available ¯_(ツ)_/¯

I think it does say something that all the other emulators with different API levels that I tried didn't have this issue. Perhaps it's related to #1237 since it only seems to be happening on API 34, and it doesn't look like the fix for that issue is applied in PlayerSurface

[oceanjules]

I'm afraid on the real device, the scaling is even more broken than emulator.

Repro for bookkeeping:

• Flashed a Pixel 8 Pro with Android 14 (of September 2024 build)
• TextureView works, SurfaceView doesn't
• Android14 with a February 2024 build works! (just like SurfaceView inside Compose AndroidView often shown stretched/cropped on API 34 #1237 specified as well)
• Scaling is all over the place, showing black screen or black strips
• Interestingly, LayoutInspector shows correct sizing for AndroidView for all ContentScale types, however on Crop or FillBounds, the Surface is not drawn properly to occupy those new constraints.

test.mov

PXL_20251013_152515190.mp4

I will try to work on the fix this week and get back to you. As you said, the syncgroup fix was only applied to PlayerView, so there isn't an immediate solution. But I just tried to use the same logic on the inner view we create inside AndroidView and we can definitely add the same logic to:

media/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/PlayerSurface.kt

Lines 88 to 95 in bfe5930

	AndroidView(
	modifier = modifier,
	factory = { createView(it) },
	onReset = {},
	update = { view = it },
	)
	view?.let { view ->

Perhaps with something like

  val createSurfaceView: (Context) -> SurfaceView = { context ->
    if (surfaceSyncGroupCompatV34 != null) {
      object : SurfaceView(context) {
        override fun dispatchDraw(canvas: Canvas) {
          super.dispatchDraw(canvas)
          surfaceSyncGroupCompatV34.maybeMarkSyncReadyAndClear()
        }
      }
    } else {
      SurfaceView(context)
    }
  }

view?.let { view ->
  DisposableEffect(view, player) {
    val listener = if (player != null && mainLooperHandler != null && surfaceSyncGroupCompatV34 != null && view is SurfaceView) {
      object : Player.Listener {
        override fun onSurfaceSizeChanged(width: Int, height: Int) {
          surfaceSyncGroupCompatV34.postRegister(mainLooperHandler, view, Runnable { view.invalidate() })
        }
      }.also {
        player.addListener(it)
      }
    } else null

    onDispose {
      listener?.let { player?.removeListener(it) }
    }
  }