Investigate support for spatial audio on Apple platforms

[rcombs]

This would require switching to AVSampleBufferAudioRenderer for audio playout. Might be tricky to account for the resulting latency.

[skrew]

VLC added it, the patch doesn't seems to be overkill to add

https://code.videolan.org/videolan/vlc/-/merge_requests/786/diffs?commit_id=744248f5ee6e2ffa205e2282036f9450025bc80f

[richtong]

please! :-)

[rcombs]

Oh, it looks like we wouldn't actually need AVSampleBufferAudioRenderer; we can just call [AVAudioSessionPortDescription setSupportsMultichannelContent] when spatialAudioEnabled is set by the system.

[tmm1]

That VLC patch is nonsense.

setSupportsMultichannelContent should be called unconditionally.

Spatial audio only works with AVSBAR.

[linust]

Any other train of thoughts on the subject or is it just hopeless?

[tbodt]

Any other train of thoughts on the subject or is it just hopeless?

It sounds really easy, you could try to do it

[linust]

Any other train of thoughts on the subject or is it just hopeless?

It sounds really easy, you could try to do it

No need to be snarky. I'm genuinely curious if this has a reasonable feasibility or if the nature of the current architecture makes Spatial Audio support on mpv as likely as (stable) Raid5 on BTRFS. Sometimes it's just a not-bloody-likely-mate situation.

[tmm1]

As mentioned several times above, this is possible only if mpv adds an audio driver that uses the AVSBAR api.

[tmm1]

I started to write an AVSBAR driver a few years ago: https://gist.github.com/tmm1/eaf6a982f967d92516915abff6794ead

I ran into a lot of bugs though, and the mpv internals around audio have changed significantly since then.

I also found that AVSBAR is incredibly buggy. For example if you use it with AirPlay speakers it is barely usable: https://github.com/tmm1/avsbartest

[tmm1]

Another approach that may or may not work: feed raw audio into AVPlayer using the CAFF format (https://developer.apple.com/library/archive/documentation/MusicAudio/Reference/CAFSpec/CAF_spec/CAF_spec.html)

I spiked a driver here, but didn't have time to investigate all the strange bugs: tmm1@96460cf

[KrisCris]

One observation - when the iOS video player nPlayer is switched to Picture in Picture mode, it uses QuickTime to decode, at which the spatial audio is enabled by default (if the video supports it)

[byGits]

I hope you'll add support for Spatial Audio soon. It's the only missing piece in making MPV/IINA perfect.

[ghost]

I started to write an AVSBAR driver a few years ago: https://gist.github.com/tmm1/eaf6a982f967d92516915abff6794ead

I ran into a lot of bugs though, and the mpv internals around audio have changed significantly since then.

I also found that AVSBAR is incredibly buggy. For example if you use it with AirPlay speakers it is barely usable: https://github.com/tmm1/avsbartest

@tmm1 I have used AVSBAR in my code and I think waiting for few seconds after first enqueuing when connected to AirPlay speakers is acceptable. Does it trouble much? Or are there other bugs with AVSBAR?

[tmm1]

I don't remember all the issues now. Some may be fixed in newer iOS, and other may not matter for your application.

[mjbaldwin]

Just wanted to +1 this, as Infuse is currently the only player that supports spatial audio (I pay for it just for that reason). I'd like to switch to IINA for its HDR support and general awesomeness, but the lack of spatial audio is holding me back.

Is the code referenced just above in KSPlayer a helpful model? The commit it references is: kingslay/KSPlayer@cf56d1c

[ArtemBernatskyy]

abandonware

[brownknightrises23]

Any update on this?

[fsafq41]

Any update on this? Thanks

[varenc]

@fsafq41 PR #11955 getting merged is seems key for mpv spatial audio support. Looks like it's waiting for a review by an mpv committer but not sure 🤷‍♀️

(though possibly a different approach using something like the VLC patch is enough)

[ShlomoCode]

VLC added it, the patch doesn't seems to be overkill to add

code.videolan.org/videolan/vlc/-/merge_requests/786/diffs?commit_id=744248f5ee6e2ffa205e2282036f9450025bc80f

VLC does not support Spatial Audio

[Akemi]

added with #11955.