Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.ogg is stuttering when play #8891

Closed
rbattaglin25 opened this issue May 1, 2021 · 15 comments
Closed

.ogg is stuttering when play #8891

rbattaglin25 opened this issue May 1, 2021 · 15 comments
Assignees
Labels

Comments

@rbattaglin25
Copy link

We noticed when the file .ogg contains the instrument "Reco Reco" the audio is stuttering in some specific devices. For example, Samsung Galaxy Tab S3 with SDK 28.

https://github.com/rbattaglin25/poc_exoplayer.git

Here is a small project using exoplayer that has this problem.

@christosts
Copy link
Contributor

I am not an audio compression/codec expert but this sounds unrelated to ExoPlayer and more related to the device's audio decoder. I tried playing a few the ogg files in the repo that you linked but I did not observe any audio stuttering.

Some ideas:

  • If you play the specific ogg files on the same devices with another player, do you still observe the issue?
  • Are you using the audio codecs that come with the device/platform? Can you try the ffmpeg extension to decode audio with ffmpeg?

@rbattaglin25
Copy link
Author

If you play the specific ogg files on the same devices with another player, do you still observe the issue?

Yes, I'm using the MediaPlayer is this PoC and it's working.

Are you using the audio codecs that come with the device/platform? Can you try the ffmpeg extension to decode audio with ffmpeg?

I'm trying to generate this file but I can't because I have to export in eight channels. Do you have any idea how to do it?

I did more tests and noticed that Xiaomi Mi9, Galaxy A50 and Moto G9 power has this problem.

In this PoC has more .ogg files without the instrument "Reco Reco" and if you play works fine .You can test this.

@christosts
Copy link
Contributor

I'm trying to generate this file but I can't because I have to export in eight channels. Do you have any idea how to do it?
I'm afraid I cannot assist with this.

At this point, I assume there's something wrong with the platform's decoder on this specific devices, and I'd advise to try ExoPlayer with the ffmpeg extension so that the player uses ffmpeg for decoding the audio files.

@krocard any other thoughts/recommendations on this issue?

@christosts christosts assigned krocard and unassigned christosts May 14, 2021
@krocard
Copy link
Contributor

krocard commented May 14, 2021

Playing your POC and our demo app with the android emulator (pixel 3XL), I could hear audio underruns.
I believe the issue comes from very high bitrate of your audio file:

$ mediainfo app/src/main/res/raw/choro_with_recoreco.ogg
Format                                   : Ogg
File size                                : 3.04 MiB
Duration                                 : 10 s 435 ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 2 440 kb/s

Audio
Format                                   : Vorbis
Bit rate mode                            : Variable
Bit rate                                 : 1 919 kb/s
Channel(s)                               : 8 channels

On low end devices like the emulator, the decoder has a very hard time decoding fast enough for real time playback. It barely manages it, but with a lot of variability. Some frames take sometime a very long time to decode, which starves the audioTrack, leading to an audible underrun.

The reason that MediaPlayer is not (as) affected compared to AudioTrack is that it uses an audio buffer twice as big. This can easily be observed by doubling MIN_BUFFER_DURATION_US to .5s:

/** A minimum length for the {@link AudioTrack} buffer, in microseconds. */
private static final long MIN_BUFFER_DURATION_US = 250_000;

@krocard
Copy link
Contributor

krocard commented May 14, 2021

I would recommend lowering the bitrate of the audio file played. 8 channel 1.9kb/s is an extremely low compression ratio (only ~35%). I guess the highest quality setting of libvorbis was used to encode them. Halving the bitrate should provide ample enough quality for playback.

@Elton1763
Copy link

We did the maximum compression of the audio file and it works on Xiaomi and Galaxy A50, but it doesn't work on Moto G (9) Power.
Do we have any possibility to adjust the Exoplayer Buffer to be similar to the MediaPlayer?

@rbattaglin25
Copy link
Author

Hello, @krocard. Can you help with the previous question?

I updated the poc with test compression audios for analysis.

@krocard
Copy link
Contributor

krocard commented Jun 29, 2021

We did the maximum compression of the audio file and it works on Xiaomi and Galaxy A50, but it doesn't work on Moto G (9) Power.

What bitrate are the audio file now? Underrun can have many causes, it is possible that the Moto G has an other issue.

Passing a custom buffer size to the DefaultAudioSink is not very convenient, but there are 2 ways:

  1. wrap the DefaultAudioSink in a FowradingAudioSink, overriding configure to change specifiedBufferSize.
  2. modify DefaultAudioSink to change MIN_BUFFER_DURATION_US, I suggest doubling it.

I would recommend using method 2 to validating that increasing the buffer size does indeed solve you issue.
If it does you can use method 1 in production to override the value without forking ExoPlayer.

@rbattaglin25
Copy link
Author

We did the way 1 and didn't work. Do you have any suggest?

@krocard
Copy link
Contributor

krocard commented Aug 11, 2021

Can you share your change? Additionally, I would still recommend trying option 2 if you have access to the device first as it is a quick way to make sure increasing the buffer indeed fixes your issue.

@rbattaglin25
Copy link
Author

I updated the project https://github.com/rbattaglin25/poc_exoplayer.git with these changes. You can help us trying option 2, we don't know how to make this.

@krocard
Copy link
Contributor

krocard commented Aug 12, 2021

You can help us trying option 2, we don't know how to make this.

Option 2 is copying DefaultAudioSink.java in your POC and changing MIN_BUFFER_DURATION_US to 500_000.

I updated the project https://github.com/rbattaglin25/poc_exoplayer.git with these changes.

I looked at the option 1, there are several issues:

https://github.com/rbattaglin25/poc_exoplayer/blob/9b5698875fb5df831908c3f483115697a4872d2f/app/src/main/java/com/ebatuque/poc_ebatuque_android/AudioServiceExoPlayer.java#L63
forwardingAudioSink.configure(format, bufferSize, outputChannels);

You should not call configure yourself, any value passed will be overwritten by ExoPlayer when playback starts. You need to subclass ForwardAudioSink to override configure to inject the buffer size.

Next the buffer size passed to configure is in bytes, not in microsecond.
You need to multiply the time by the frequency, the channel size and the frame size:

      pcmFrameSize = 2; // assuming you have not enabled float output
bufferSize = pcmFrameSize * 500_000 * format.sampleRate * format.channelCount / C. MICROS_PER_SECOND ;

Though this solution 1 is fragile, it will break in passthrough.
Far better would be to add an accessor to DefaultAudioSink to change the MIN/MAX buffer duration. Alternatively, we might want to increase MIN_BUFFER_DURATION_US to 500ms to match media player.

@krocard
Copy link
Contributor

krocard commented Aug 12, 2021

Thinking more about it, given that we want ExoPlayer to be a seamless upgrade over MediaPlayer, and that customizing the BufferSize requires expert knowledge as well fragile code. I believe we should align ExoPlayer to MediaPlayer.
I have prepared the internal cl [cl/390364027] to make such transition. Nevertheless it might not be immediately merged at we need to check for regressions.

@krocard
Copy link
Contributor

krocard commented Nov 24, 2021

The plan is now to allow ExoPlayer users to customize the calculation of the AudioTrack buffer size. I will report the exact API went it is merged.

icbaker pushed a commit to androidx/media that referenced this issue Dec 10, 2021
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.

The new class is easily configurable and highly tested.
Manual test was used to catch any regression.

google/ExoPlayer#8891

PiperOrigin-RevId: 415268938
icbaker pushed a commit to androidx/media that referenced this issue Dec 10, 2021
*** Original commit ***

Make audio track min buffer size configurable.

Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.

The new class is easily configurable and highly tested.
Manual test was used to catch any regression.

google/ExoPlayer#8891

***

PiperOrigin-RevId: 415469179
ojw28 pushed a commit that referenced this issue Dec 10, 2021
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.

The new class is easily configurable and highly tested.
Manual test was used to catch any regression.

#8891

PiperOrigin-RevId: 415268938
ojw28 pushed a commit that referenced this issue Dec 10, 2021
*** Original commit ***

Make audio track min buffer size configurable.

Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.

The new class is easily configurable and highly tested.
Manual test was used to catch any regression.

#8891

***

PiperOrigin-RevId: 415469179
andrewlewis pushed a commit to androidx/media that referenced this issue Jan 28, 2022
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.

The new class is easily configurable and highly tested.
Manual test was used to catch any regression.

google/ExoPlayer#8891

PiperOrigin-RevId: 424602011
andrewlewis pushed a commit that referenced this issue Jan 28, 2022
Move the code in its own class as DefaultAudioTrack
is getting very big. It also help for testability.

The new class is easily configurable and highly tested.
Manual test was used to catch any regression.

#8891

PiperOrigin-RevId: 424602011
@krocard
Copy link
Contributor

krocard commented May 13, 2022

The size of the audio track buffer is now configurable. Setting the min buffer duration to 500ms as should allow to have the same underrun behaviour as MediaPlayer.

* Sets the minimum length for PCM {@link AudioTrack} buffers, in microseconds. Default is
* {@value #MIN_PCM_BUFFER_DURATION_US}.
*/
public Builder setMinPcmBufferDurationUs(int minPcmBufferDurationUs) {

@krocard krocard closed this as completed May 13, 2022
@google google locked and limited conversation to collaborators Jul 13, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

5 participants