wasapi Dolby THD channel confusion can we use LAV Filters

[netExtra]

Important Information

Latest mpv version
Windows 10
Any mpv source
Nvidia RTX GPU

Reproduction steps

Play any THD Atmos movie/file/demo, rewind and replay

Expected behavior

Sounds come out of the same channels as before.

Actual behavior

Sometimes a sound from the right channel comes out of the left channel. Sometimes this happens without rewinding. It's not consistent but I can get it to happen regularly. It's been a problem for a long time with various different mpv/ffmpeg source. I've tried to reproduce the same issue with LAV but I can't. I have changed the heading because this is likely to be an issue with THD rather than specifically with Atmos.

I just need to know if it is somehow possible to use LAV filters for audio passthough?

[jeeb]

Short answer: no

It'd help to know how LAV supposedly would be handling this better in technical manner given that passthrough would be actual bit stream instead of decoded results, it sounds really weird if stereo channels can get flipped. LAV is also based on its own fork of FFmpeg for reading the content, not sure if it has its own SPDIF writer or it utilizes the one from its FFmpeg.

[ruihe774]

Can ffmpeg decodes Dolby Atmos? I don't think so. IIRC ffmpeg can only decode Dolby Digital Plus. Dolby Atmos can be decoded by Dolby Digital Plus compatible decoders but some info will be lost (e.g. 7.1 -> 5.1). If you need to play Dolby Atmos as is you may need SPDIF passthru.

If there is issue decoding Dolby Atmos with ffmpeg's Dolby Digital Plus decoder you may report it to ffmpeg, IMO.

[netExtra]

Can ffmpeg decodes Dolby Atmos?

I'm talking about passthrough. Which as @jeeb pointed out is why this is a weird issue. But it is very repeatable.

[kasper93]

There might be multiple reasons for flaky behavior like that and at the same time impossible for us to diagnose remotely. You need to compare stream parameters and possibly data that goes out of mpv and LAV. Both project are opensource so it should be easy to debug.

[ruihe774]

BTW can you provide your log file?

[netExtra]

BTW can you provide your log file?

I didn't bother because I know this is likely to be more of an ffmpeg issue. Also logs are set to debug mode but I didn't see anything about individual channels. Just stuff about the stream opening, pausing, resetting and eventually closing.

[ruihe774]

BTW can you provide your log file?

I didn't bother because I know this is likely to be more of an ffmpeg issue. Also logs are set to debug mode but I didn't see anything about individual channels. Just stuff about the stream opening, pausing, resetting and eventually closing.

I want to know whether WASAPI SPDIF passthru is working properly. If you are assured that it is a issue of ffmpeg, you can report it to the upstream.

[netExtra]

BTW can you provide your log file?

I didn't bother because I know this is likely to be more of an ffmpeg issue. Also logs are set to debug mode but I didn't see anything about individual channels. Just stuff about the stream opening, pausing, resetting and eventually closing.

I want to know whether WASAPI SPDIF passthru is working properly. If you are assured that it is a issue of ffmpeg, you can report it to the upstream.

Hi
@ruihe774
It's not the right channel that goes to the left. It's the surround rear channels end up in the surround left.
The logs attached below record me playing the file and rewinding. After the error occurs I stop the file.

mpv.log

Thanks

[ruihe774]

The logs attached below record me playing the file and rewinding. After the error occurs I stop the file.
mpv.log

SPDIF activated properly. Looks like an issue of ffmpeg.

BTW it'd better say "Dolby Atmos with TrueHD". It's quite different from Dolby Atmos with E-AC-3.

LAV Filters have special handling with TrueHD. (You can find it in its changelog.) Maybe this is the cause of different behaviors.

[netExtra]

The logs attached below record me playing the file and rewinding. After the error occurs I stop the file.
mpv.log

SPDIF activated properly. Looks like an issue of ffmpeg.

BTW it'd better say "Dolby Atmos with TrueHD". It's quite different from Dolby Atmos with E-AC-3.

LAV Filters have special handling with TrueHD. (You can find it in its changelog.) Maybe this is the cause of different behaviors.

I'm using a version of ffmpeg with a fix which fixes a previous "fix" which actually broke THD Atmos. I had the same issue with the previous broken ffmpeg version as well. Since the the broken ffmpeg hasn't been fixed for some time I don't hold out much hope for an issue as flakey as this. But thanks anyway :)

[starchivore]

If I weren't mistaken, @mitzsch should have mentioned some like that?

#13794

In other words, not sure if his fork were considered a workaround or not?

https://github.com/mitzsch/mpv-winbuild/releases
https://forums.plex.tv/t/dolby-truehd-passthrough-modified-mpv-build/802742

BTW, supposedly Dolby TrueHD should have multiple flavors and therefore just wondering if only object-based Atmos tracks were affected?

https://www.reddit.com/r/hometheater/comments/zh8ufk/512_setup_dolby_truehd_vs_dolby_atmos_which_one/

Or more like both channel-based and object-based formats are actually behaving the same way?

[netExtra]

If I weren't mistaken, @mitzsch should have mentioned some like that?

#13794

In other words, not sure if his fork were considered a workaround or not?

https://github.com/mitzsch/mpv-winbuild/releases https://forums.plex.tv/t/dolby-truehd-passthrough-modified-mpv-build/802742

BTW, supposedly Dolby TrueHD should have multiple flavors and therefore just wondering if only object-based Atmos tracks were affected?

https://www.reddit.com/r/hometheater/comments/zh8ufk/512_setup_dolby_truehd_vs_dolby_atmos_which_one/

Or more like both channel-based and object-based formats are actually behaving the same way?

Atmos is by definition object based. There's are two types of Atmos. When I stated Atmos in the heading I was asked to clarify which type of Atmos. This is about THD Atmos but thinking about it, it probably also applies to THD. The @mitzsch fork fixes the broken THD ffmpeg code and I am already using it. But this is a different issue which I think has been around since before the fix. But it might be worth confirming.

[starchivore]

https://forum.doom9.org/showthread.php?p=1817199#post1817199

Is it possible to make your version of mpv-support video/audio directshow filters?

https://forum.doom9.org/showthread.php?p=1817207#post1817207

mpv probably won't support dshow, for some time I thought about having two playback engines, like dshow and libmpv, it could be done and might make sense but only if you have a lot time and interest, in StaxRip normally I try to support different encoders, it supports both avisynth and vapoursynth equally well and I'm very happy with the decision to support vapoursynth. Doing a player with two playback engines is much more difficult I think and most players don't get it right even with only one. I don't have enough time and interest for both so I decided to focus on one and make the best out of it.

LAV filters are obviously directshow filters and therefore we could only borrow the implementation, pretty much like what Kodi was doing to deal with TrueHD (i.e. Metadata-enhanced Audio Transmission or MAT) passthrough:

Metadata-enhanced Audio Transmission
https://avproedge.com/blogs/news/a-deep-dive-into-dolby-mat

Possible new Seamless branching Issues
https://forum.makemkv.com/forum/viewtopic.php?f=6&t=33890

TrueHD Atmos Dropouts/Cutouts on Kodi 20.5 (assuming only seamless branched sources) #24944
xbmc/xbmc#24944

[Audio] TrueHD rework - totally new MAT packer implementation #24984
xbmc/xbmc#24984

Based on LAVFilters: https://github.com/Nevcairiel/LAVFilters/blob/master/decoder/LAVAudio/BitstreamMAT.cpp
with some optimizations.

Release Audio Passthrough IEC - TrueHD fix/workaround - Testing build
https://forum.kodi.tv/showthread.php?tid=371292

Bug TrueHD Atmos Dropouts/Cutouts on Kodi 20.5 (assuming only seamless branched sources)
https://forum.kodi.tv/showthread.php?tid=376726

TrueHD passthrough Test Builds - New MAT Packer implementation
https://forum.kodi.tv/showthread.php?tid=377117

While current MAT packer for TrueHD works really very well, I have found some rare cases (0.1%) that may produce non-compliant bitstream, especially related to seamless branching Blu-Ray sources (or resultant MKVs remuxes).

The new implementation is based on LAVFilters implementation with some optimizations and small changes. Then is totally different from current FFmpeg based implementation...

Hopefully we'll find a way to get that implemented and see how it goes, many thanks.

[ruihe774]

https://forum.doom9.org/showthread.php?p=1817199#post1817199

Is it possible to make your version of mpv-support video/audio directshow filters?

https://forum.doom9.org/showthread.php?p=1817207#post1817207

mpv probably won't support dshow, for some time I thought about having two playback engines, like dshow and libmpv, it could be done and might make sense but only if you have a lot time and interest, in StaxRip normally I try to support different encoders, it supports both avisynth and vapoursynth equally well and I'm very happy with the decision to support vapoursynth. Doing a player with two playback engines is much more difficult I think and most players don't get it right even with only one. I don't have enough time and interest for both so I decided to focus on one and make the best out of it.

LAV filters are obviously directshow filters and therefore we could only borrow the implementation, pretty much like what Kodi was doing to deal with TrueHD (i.e. Metadata-enhanced Audio Transmission or MAT) passthrough:

Metadata-enhanced Audio Transmission https://avproedge.com/blogs/news/a-deep-dive-into-dolby-mat

Possible new Seamless branching Issues https://forum.makemkv.com/forum/viewtopic.php?f=6&t=33890

TrueHD Atmos Dropouts/Cutouts on Kodi 20.5 (assuming only seamless branched sources) #24944 xbmc/xbmc#24944

[Audio] TrueHD rework - totally new MAT packer implementation #24984 xbmc/xbmc#24984

Based on LAVFilters: https://github.com/Nevcairiel/LAVFilters/blob/master/decoder/LAVAudio/BitstreamMAT.cpp
with some optimizations.

Release Audio Passthrough IEC - TrueHD fix/workaround - Testing build https://forum.kodi.tv/showthread.php?tid=371292

Bug TrueHD Atmos Dropouts/Cutouts on Kodi 20.5 (assuming only seamless branched sources) https://forum.kodi.tv/showthread.php?tid=376726

TrueHD passthrough Test Builds - New MAT Packer implementation https://forum.kodi.tv/showthread.php?tid=377117

While current MAT packer for TrueHD works really very well, I have found some rare cases (0.1%) that may produce non-compliant bitstream, especially related to seamless branching Blu-Ray sources (or resultant MKVs remuxes).
The new implementation is based on LAVFilters implementation with some optimizations and small changes. Then is totally different from current FFmpeg based implementation...

Hopefully we'll find a way to get that implemented and see how it goes, many thanks.

Thank you for listing relevant materials. They really cast light on THD Atmos passthru.

Nevertheless, mpv depend solely on ffmpeg for audio stream parsing and demuxing. If the upstream does not have support on it, there is little we can do in mpv at this time. In theory, it is possible to implement a specialized parser & demuxer in audio/decode/ for these proprietary codecs; but it requires considerable effort.

[netExtra]

Well there is already a fix that works > 99% of the time for THD passthrough cutouts. I do not know why it has never been incorporated into the main ffmpeg code base. The New MAT Packer implementation reminds me of the ffmpeg "fix" which broke THD which is weird.

[netExtra]

As for my issue, after some additional testing, it seems to be much harder to replicated in exclusive mode. I tend not to use E mode because can cause sound config issues with other non exclusive devices. But I understand a little more than I did the last time I tried it so it should be OK.

[zhouxinghong]

👍

[mitzsch]

Okay, also peeking into this issue.

Somehow I can not believe that audio channels can be swapped by the source with passthrough enabled and working (= the AVR decodes the bitstream in a nondistorted way) decoding. In my understanding of passthrough, if audio is bitstreamed and the avr decodes it fine (in my definition not mangled audio) it works as intended. However, when interrupting the stream (like fast forward, skip, rewinding) things can get hairy - but it seems the decoding device is the culprit in that case.

I can reliably reproduce a very weird issue on my avr. (with mpv or lavfilters)
When bit streaming DTS-HD MA 7.1 sometimes the AVR only sees DTS-HD MA 5.1 after I skipped too aggressively. My understanding is that the source is sending the data in an overflow manner and confuses the decoder software/hardware. Pausing the stream and starting playback "fixes" the problem - the avr shows DTS-HD MA 7.1 again.

Reinitializing the audio output after skipping also prevents this bug from happening. In my understanding and testing my BluRay player does so - after skipping audio returns slightly delayed. (= because of the reinit of the audio output)

To mimic this behavior I wrote a Lua script.
reinitonseek.lua.txt
All it does is changing the audio stream to none, wait 0.4 seconds, and then change it back to the initial audio track after seeking.
Please try this script. (change the file extension back to .lua - GitHub does not support uploading .lua files...)
Does it fix the issue?

Other questions I have:

Does the issue also arise with TrueHD (non-Atmos) files?
Does the issue also arise with my old-trueHD builds? => https://github.com/mitzsch/mpv-winbuild/actions/runs/9017168715

[netExtra]

Okay, also peeking into this issue.

Somehow I can not believe that audio channels can be swapped by the source with passthrough enabled and working (= the AVR decodes the bitstream in a nondistorted way) decoding. In my understanding of passthrough, if audio is bitstreamed and the avr decodes it fine (in my definition not mangled audio) it works as intended. However, when interrupting the stream (like fast forward, skip, rewinding) things can get hairy - but it seems the decoding device is the culprit in that case.

I can reliably reproduce a very weird issue on my avr. (with mpv or lavfilters) When bit streaming DTS-HD MA 7.1 sometimes the AVR only sees DTS-HD MA 5.1 after I skipped too aggressively. My understanding is that the source is sending the data in an overflow manner and confuses the decoder software/hardware. Pausing the stream and starting playback "fixes" the problem - the avr shows DTS-HD MA 7.1 again.

Reinitializing the audio output after skipping also prevents this bug from happening. In my understanding and testing my BluRay player does so - after skipping audio returns slightly delayed. (= because of the reinit of the audio output)

To mimic this behavior I wrote a Lua script. reinitonseek.lua.txt All it does is changing the audio stream to none, wait 0.4 seconds, and then change it back to the initial audio track after seeking. Please try this script. (change the file extension back to .lua - GitHub does not support uploading .lua files...) Does it fix the issue?

Other questions I have:

Does the issue also arise with TrueHD (non-Atmos) files? Does the issue also arise with my old-trueHD builds? => https://github.com/mitzsch/mpv-winbuild/actions/runs/9017168715

Hi @mitzsch

Interesting points. Did you see my last post? Using exclusive mode seems to mitigate the issue and I wasn't able to repeat the same problem. So I didn't bother to test with an mpv build without the ffmpeg fix. Also, I do wonder if using audio-stream-silence might help when not in exclusive mode?

[mitzsch]

So I didn't bother to test with an mpv build without the ffmpeg fix.

Well, it would help us to understand this problem and narrow it down to what may cause this. So please test:

1. the lua script
2. the old trueHD implementation
3. Non-Atmos TrueHD files

Thanks...

Edit#
Working audio in exclusive mode is interesting, but again please test/answer the questions above.

[netExtra]

So I didn't bother to test with an mpv build without the ffmpeg fix.

Well, it would help us to understand this problem and narrow it down to what may cause this. So please test:

1. the lua script
2. the old trueHD implementation
3. Non-Atmos TrueHD files

Thanks...

Edit# Working audio in exclusive mode is interesting, but again please test/answer the questions above.

When I was younger, sometimes my teeth used to stop hurting when I arrived at the dentist surgery. Why is this relevant? Because I can't get the damn issue to happen again. I turned off exclusive mode on Windows and in mpv and it still works perfectly now. When I did try your script I don't think it worked for me. All I saw in the logs was that it was looking for resetoninit.conf.

Anyway this has been interesting. If nothing else it's a reminder that Dolby THD is a problem, especially with ffmpeg. This was repeatable at the time but now it isn't. Maybe something in my PC or receiver reset once I set it to exclusive mode and now it works exclusive or not. So unless anyone has any objections I think I should close this ticket?

[mitzsch]

When I was younger, sometimes my teeth used to stop hurting when I arrived at the dentist surgery.

Those are the patients we love. :D
(I´m a dentistry student ^^)

Joke aside.

Because I can't get the damn issue to happen again.

This would also hold up with my assumption that's its not an mpv issue and probably a decoder problem.

When I did try your script I don't think it worked for me.

On my end it works just fine.
Have you changed the file extension back to .lua and loaded it with -script="path\to\reinitonseek.lua"?

So unless anyone has any objections I think I should close this ticket?

Closing the issue and reopening it in case it re-appears is probably the best option. If it re-appears please also test the lua script (it should work as described above)

[netExtra]

When I did try your script I don't think it worked for me.

On my end it works just fine. Have you changed the file extension back to .lua and loaded it with -script="path\to\reinitonseek.lua"?

So unless anyone has any objections I think I should close this ticket?

Closing the issue and reopening it in case it re-appears is probably the best option. If it re-appears please also test the lua script (it should work as described above)

I reverted the extension to .lua and ust added the script to my Scripts folder. MPV would not be asking for the .conf otherwise :) Anyway yes I will close it for now. Thanks :)