mac-virtualcam: Fix frame sharing and colourspace issues #7403
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This issue primarily seems to affect Intel-based Macs without unified memory: While an IOSurface is supposed to be shared via a mach port with other processes, each process needs to lock the surface during access (as is common with other shared resources). Apple Silicon-based Macs seem to be less affected as a switch between GPU and CPU memory (which can happen dynamically for IOSurfaces) would still point to the same unified memory.
The root cause of the issue is `swscale` dropping the second plane of biplanar pixel data, resulting in an "incomplete" frame being fed to the CVPixelBuffer. As CVPixelBuffers have dedicated support for full range colour, use these directly, which improves performance even further (as any conversion for full range data is avoided as well). To ensure that OBS does not implicitly enable conversion via `swscale` a video conversion struct needs to be set in any case, ensuring that the output range and colourspace match the output configuration.
PatTheMav
added
Bug Fix
|
Kudos to @ccgus for dropping the important hint on his blog: https://shapeof.com/archives/2017/12/moving_to_metal_episode_iii.html. |
|
Per off-thread conversation with @PatTheMav and @gxalpha , this is good to merge as-is, so I'm merging it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes an issue with empty IOSurface references on Intel-based Macs and also broken output when OBS is set to deliver full colour range output.
Motivation and Context
The plugin was missing necessary IOSurface locks to ensure that modifications to the underlying memory reference made by the OS do not occur when client code accesses the image data.
On machines without unified memory this can lead to IOSurfaces pointing to GPU memory even though the data has been moved to CPU memory by the OS (to avoid unnecessarily large copy operations) and a PixelBuffer created from the IOSurface won't contain data.
In addition to the issue above, FFmpeg's
swscaleseems to have issues with converting images with full colour ranges to limited colour ranges, resulting in mangled frame data inside OBS.As macOS natively supports partial range as well as as full ranges for pixel buffers, use these directly, avoiding costly colour space conversions for even more use cases than before - as this skips swscale entirely, the issue doesn't occur anymore.
Fixes #7333
Possibly fixes #7171
How Has This Been Tested?
Tested with macOS 12.6 on Intel-based and Apple-Silicon-based Macs.
Types of changes
Checklist: