Getting some glitches due to lack of sufficient encoder pre and post roll.

[davidnaas]

First of, I know that this is an old repo but I'm hoping for the best..

I tried to adapt the example to a more real world implementation but can't quite get it to work. For the most part it plays back perfectly but there's still a few clicks that I can't seem to get rid of.

I chopped up an already existing mp3 file like this:

ffmpeg -i original.mp3 -f segment -segment_time 3 -c copy out%03d.mp3

I'm then serving 100 of these files of a small node server. However, I get the same three clicks occurring at the same time mark every time. I also noticed when trying to debug it that it's actually only two frames where the frontPadding and endPaddig has a value larger than 0. (Is there anyway I can confirm that clicks occur for the frames with a value for the padding?)

It's pretty much just copy paste from your code and admittedly, I have no idea what I'm doing.

[theicfire]

I've seen clicking happen even when encoding with LAME and the encoding headers are all showing up. The clicking is very slightl, and is particularly apparent when playing a 1khz sine wave. Otherwise it's not too noticeable

[dalecurtis]

Whoops, sorry don't have notifications turned on for issues, so just saw this. I think splitting the mp3 file like you're doing c#0 doesn't need trimming.

You're just remuxing the content into multiple mp3 files versus doing separate encodings. In which case I think you can just append the segments sequentially without specifying the append window and it should sound correct correct.

While I split a single file up for the purposes of this demo, in practice most real use cases will just have different pieces of content they need to stitch together.

If you're able to reproduce any issues in Chrome, please file a bug at https://crbug.com/new with the samples and I can take a look.

[theicfire]

I'd be happy to make a minimal test case with LAME. I'm encoding separate segments of a 1khz sine wave. Is cbrug.com/new a typo?

[dalecurtis]

Err, yes, sorry. https://crbug.com/new

[theicfire]

Turns out the demo with no modifications already has the glitches I'm seeing. Here's a video where the audio glitch happens when I run this demo locally: https://www.loom.com/share/93d86c58f86a4b7486fe190078fd004a

[theicfire]

I hypothesize if the audio is just a sine wave the glitch will be even more noticeable

[dalecurtis]

Hmm, can't hear it in your demo video or locally on my similarly configured mac (10.15.3) over speakers or with isolating ear buds. What audio hardware are you listening on? Chrome's implementation should match the wave forms that are on that page, but would have to rerun that analysis to see for sure.

[dalecurtis]

Will see about generating a sine wave test, but if you have one ready to go that'd be helpful.

[theicfire]

haha oh man, I tried it both on my external speakers and some $10 earbuds and I can hear it. Yeah I'll make a sine wave demo

[dalecurtis]

I added a sine demo locally and could hear the glitch there. Will take a look to see what's happening.

[dalecurtis]

Not obvious what's going wrong, the windows and append times all look right so will need to look at whats being rendered at the output level.

[dalecurtis]

Seems to be some discrete change in the signal at the cut point, it's not clear if this something going wrong with our splicing, or an issue with the decoding.

[theicfire]

I notice when I put sine waves side by side in Ableton Live, the WAV files can be put together perfectly, but I can't find any way to put LAME encoded mp3 files together without a glitch. This could be the decoder or the encoder?

[dalecurtis]

Yes that's definitely possible too. Do you hear clicking in the AAC version at the bottom of the demo page?

[theicfire]

Woah, no I don't! The demo under "conclusion" seems perfect

[theicfire]

Is there something inherently wrong with mp3 encoding that prevents it from getting perfect gapless playback? That is, can no audio streaming platform (soundcloud, youtube, skype) use mp3?

This is where I don't know much about audio. I took the mp3 approach because I was having ffmpeg trouble getting AAC to work.

I also noticed using the Lame library seemed much simpler than figuring out how to programmatically (and in a low latency way) use something like afconvert and mp4box, but I could send the gap information through some other means.

[theicfire]

I had run these tests previously which makes me think there's something off with mp3.

The following screenshots are zoomed into a transition of a sine wave, from two segments of audio.

If I convert the raw to .wav and put them side by side in ableton, the issue is audibly perfect
Screenshot of the transition

However, if I convert to mp3 in LAME, and then convert back to wav in ffmpeg and put them side by side, there is a glitch in the audio
Screenshot of the transition

There's similarly a glitch when going from PCM -> ffmpeg mp3 -> wav
Screenshot of the transition

[dalecurtis]

I suspect lame isn't providing enough preroll or postroll to get the decoder into steady state. Whereas the AAC encoding uses 2112 and 960 for pre and post respectively. I suspect if you upped the pre and post roll for the mp3 encoding to 1152/1152 it would be correct; not sure if there's a lame option for that though.

[theicfire]

Mm, that sounds about right. Thought I can't find anything on google relating to "mp3 preroll postroll", or "pre-roll" or "pre roll" etc. Maybe for now it's worth updating the post that mp3 isn't perfect (given the tools that have been tried) and it's not recommended for gapless?

[dalecurtis]

Yeah I believe my article has a lot of the only documentation about this stuff out there unfortunately. You can find it in the code for encoders, decoders, and various bug reports, but that's about it.

I'll see about posting an update here, though I think most people find this through the Google Developers article that I'm not sure I can update anymore.

[theicfire]

I see 👍 thanks a bunch for the help! I'll be trying out AAC or Opus.

P.S. someone just mentioned to me that I might be able to solve gapless playback without MSE -- I can instead decode in webassembly and pipe the PCM to https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletNode. It's incredible how powerful browsers are these days.

[dalecurtis]

Yes that's definitely another option, you have to implement the full playback pipeline though. We're also working on WebCodecs and perhaps support for audio/wav in MSE (for WASM decoders) which might help out here.

https://github.com/WICG/web-codecs/blob/master/explainer.md
https://chromium-review.googlesource.com/c/chromium/src/+/1702789

[theicfire]

I hadn't heard of these! The limitations of MSE for low latency mode (the bottom of the second link) speak to the heart. I now know I'm not crazy and doing everything in some weird way

[dalecurtis]

https://github.com/WICG/media-latency-hint/blob/master/explainer.md might be in your interest too. It's another thing we're working on.