Bad audio quality when transcoding from AAC to Opus

[AseCoder]

Question

I am streaming an OBS output and in order to listen to the stream in a browser using WebRTC the audio codec of the stream must not be AAC. I tried to implement the solution from this issue. This is my implementation:

paths:
  fin:
  mystream:
    runOnReady:
      ffmpeg -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -vcodec copy -c:a libopus -f rtsp rtsp://localhost:$RTSP_PORT/fin
    runOnReadyRestart: yes

The results are as follows:

• The stream has audio
• The audio is crackly and distorted
• The console spits out, in yellow text, Stream #0: not enough frames to estimate rate; consider increasing probesize

I have tried increasing the probesize. The ffmpeg command now looks like this:
ffmpeg -probesize 10M -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -vcodec copy -c:a libopus -f rtsp rtsp://localhost:$RTSP_PORT/fin

When I try running that the console spits out Non-monotonous DTS in output stream 0:1; previous: 485588, current: 481212; changing to 485589. This may result in incorrect timestamps in the output file. many times.

I then tried regenerating the DTS with -fflags +igndts as suggested by the top answer to this. After that the bad audio quality persists and the only warning in the console is No bit rate set. Defaulting to 96000 bps.

How in the world do I transcode the audio without it becoming unintelligible?

MediaMTX v1.3.1
ffmpeg version 5.1 essentials
log.txt

[AseCoder]

I just tested that command on a normal video file and it worked fine. It spat out the bitrate warning but the audio quality was fine.

[aler9]

Hello, try setting the channel count and the bitrate too:

paths:
  fin:
  mystream:
    runOnReady:
      ffmpeg -i rtsp://localhost:$RTSP_PORT/$RTSP_PATH -vcodec copy -ac 2 -c:a libopus -b:a 128k -f rtsp rtsp://localhost:$RTSP_PORT/fin
    runOnReadyRestart: yes

[AseCoder]

Hi, thanks for your reply. Using the command you suggested, the console says

[aac @ 000001a4767a9a40] Input buffer exhausted before END element found
[aist#0:1/aac @ 000001a4721e40c0] Error submitting packet to decoder: Invalid data found when processing input
[aac @ 000001a4767a9a40] Reserved bit set.
[aac @ 000001a4767a9a40] Prediction is not allowed in AAC-LC.
[aist#0:1/aac @ 000001a4721e40c0] Error submitting packet to decoder: Invalid data found when processing input

once and

2023/12/07 16:05:43 WAR [RTSP] [session 308f2c1a] write queue is full
[rtsp @ 000001a471670800] max delay reached. need to consume packet
[rtsp @ 000001a471670800] RTP: missed 1 packets

many times. On another run I got this:

[aac @ 000001e188d4ca00] Input buffer exhausted before END element found
[aist#0:1/aac @ 000001e18d858e80] Error submitting packet to decoder: Invalid data found when processing input
[aac @ 000001e188d4ca00] TYPE_FIL: Input buffer exhausted before END element found
[aist#0:1/aac @ 000001e18d858e80] Error submitting packet to decoder: Invalid data found when processing input
[libopus @ 000001e188ee0b00] Queue input is backward in time
[rtsp @ 000001e189285040] Non-monotonic DTS in output stream 0:1; previous: 479720, current: 475401; changing to 479721. This may result in incorrect timestamps in the output file.
[rtsp @ 000001e189285040] Non-monotonic DTS in output stream 0:1; previous: 479721, current: 476345; changing to 479722. This may result in incorrect timestamps in the output file.
[rtsp @ 000001e189285040] Non-monotonic DTS in output stream 0:1; previous: 479722, current: 477289; changing to 479723. This may result in incorrect timestamps in the output file.
[rtsp @ 000001e189285040] Non-monotonic DTS in output stream 0:1; previous: 479723, current: 478281; changing to 479724. This may result in incorrect timestamps in the output file.
[rtsp @ 000001e189285040] Non-monotonic DTS in output stream 0:1; previous: 479724, current: 479225; changing to 479725. This may result in incorrect timestamps in the output file.

The audio is still distorted. Here is a sample of the streams audio. I also tried setting the audio sources to mono instead of stereo in OBS, but that didn't help either.

Do you have any other ideas that I could try? Thanks for your help.

[AseCoder]

@aler9 Can you help me?

I updated MediaMTX to 1.4.0, ffmpeg to 6.1 full and OBS to 30. The audio is still distorted.

[AseCoder]

#2824 suggests that the distortion is caused by interruptions in the audio. I hypothesise that the interruptions are more pronounced and frequent in my case because my source stream comes from OBS instead of a stream published on the internet, which means that the individual "stream packets" are much shorter, and libopus is somehow failing to stitch them together.

[BuildMineSurvive]

@AseCoder I found a solution that works for me. Adding -async 50 to my ffmpeg command fixed the audio problems. What I'm guessing, is that the buffer is just too small and it's underflowing and losing audio.

This flag does technically add 50ms delay to your audio. But 50ms is not really noticeable, but test out smaller numbers and see if it still works reliably for your setup.

[AseCoder]

Yup, that works for me too, thanks!

However, I found an even better solution which eliminates the need for remuxing the stream.

[BuildMineSurvive]

True! I've used opus through SRT and that indeed works, but with my setup I'm using the sever as infrastructure for video transport, and the webrtc is just for monitoring. Some of the programs I use only support AAC so I have to transcode audio regardless unfortunately.

[aler9]

Codec compatibility is still madness - my advice is to switch to patent-free codecs like AV1 for video and Opus for audio in all new projects. You won't have any problem them.

[jpetazzo]

I ran into a very similar issue with the following setup:

USB audio interface }
                    } ===> ffmpeg ===> MediaMTX ===> client browsers
OBS virtual webcam  }

I'm not transcoding: ffmpeg is encoding directly to Opus, streaming to MediaMTX over RTSP, which then serves the stream over HLS and WebRTC.

Audio is fine over HLS, but garbled over WebRTC, and it sounds exactly like the audio sample that was posted above - i.e., micro-interruptions / stutters.

The -async 50 option didn't help.

The problem doesn't show up when I stream an audio test file (instead of acquiring live audio with the USB interface).

Now, interestingly, I'm using ALSA to acquire audio with ffmpeg. If I switch to pulseaudio, the problem disappears (yay!) but there is a noticeable audio lag (boo!). The audio lag is independent from MediaMTX. It was already noticeable with my previous streaming stack (based on NGINX RTMP), and it's the reason why I was using ALSA (which had no lag).

I'll see if I can find other ways to address the audio lag, but I thought I'd post these findings here, just in case more folks have a similar problem!