When playing a video containing AAC at double speed with loop mode, seeking to the position near the end will cause stuck

[captain81]

Version

Media3 main branch

More version details

When playing a video containing AAC at double speed, seeking to the position near the end will cause stuck

Devices that reproduce the issue

all

Devices that do not reproduce the issue

NA

Reproducible in the demo app?

Yes

Reproduction steps

1.When use media3 demo playing a video at double speed with loop mode
2. drag the progress bar，seeking to the position near the end(5-second video, seek to the 4-second position)

Expected result

plays successfully

Actual result

play stuck

Media

I have send video 055.mp4 test file to android-media-github@google.com

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue. #3123

[captain81]

This issue is related to AAC/MP4A audio. If you use the command ffmpeg -i input.mp4 -an -c:v copy output.mp4 to remove the audio tracks, the issue of seeks causing playback stuttering will no longer occur.

[captain81]

The possible reason is that during loop playback, multiple sampleQueues are loaded simultaneously. The sum of the buffers of these multiple sampleQueues, getTotalBufferBytesAllocated(playerId), is 33,292,288, which exceeds getTargetBufferBytes(playerId) of 32,768,000. Therefore, targetBufferSizeReached is true, and no more data will be loaded into the sampleQueue, resulting in abnormal playback termination near the end. However, it is strange that each source will terminate abnormally when seeking to a position about 1 second before the end, but not when seeking to other positions
However, I'm not sure why the sampleQueue for MediaPeriodHolder's playing stops loading, but the sampleQueue for loading and reading continues to load

LOG:
getTotalBufferBytesAllocated(playerId): 33292288 targetBufferSizeReached true getTargetBufferBytes(playerId) 32768000
shouldContinueLoading: false bufferedDurationUs 13083900

[guczOasis]

Hi @tonihei ，any update for this issue? We found the same problem. Please help to check it.
Thanks.

[captain81]

Hi @tonihei, When seeking the 055.mp4 to around 4180ms, the video playback stuck. The reason is that the positionUs of getFrameReleaseAction() is no longer updated. It may be that the remaining audio is too short, about 1 second, and the positionUs is no longer updated, resulting in positionUs == initialPositionUs. getFrameReleaseAction() keeps returning FRAME_RELEASE_TRY_AGAIN_LATER, so the subsequent video frames cannot be sent to the display, causing the video playback to stuck. How can I solve the problem of positionUs not being updated when seeking close to the end of the file? How can I solve the problem of video playback freezing in this situation?

if (!started || positionUs == initialPositionUs) {
return FRAME_RELEASE_TRY_AGAIN_LATER;
}

[captain81]

@marcbaechinger Please help to check it.

[tonihei]

Thanks for the reproduction steps, I can the same issue as well. It's essentially caused by a kind of three-way deadlock where the audio track doesn't want to start playback because it doesn't have enough data, and the player doesn't want to continue providing data from the next repeated item because the video isn't finished yet, and the video can't start because the audio playback hasn't started yet. This is in fact a long standing issue, for example reported in google/ExoPlayer#6459. Note that the speed adjustment makes the problem slightly more visible because the data written to the audio track is much shorter.

The good thing is that we've been already working on a fix for this recently, but it's fairly involved because it changes some core mechanics of the player. @microkatz assigning to you since you are working on the fix that allows the streams to advance reading independently.