Release Candiate (RC) suffix on version number breaks macOS DAL VirtualCam plugin version comparison

[wookayin]

Platform

Operating system and version: MacOS 11.0 Big Sur (a M1 mac)
OBS Studio version: 26.1.0 rc2

Expected Behavior

The virtual camera plugin should work -- the OBS virtual cam should appear in the camera list.

Current Behavior

The virtual camera DAL plugin shipped with and installed from OBS cannot be used in many applications. The camera list does not show OBS Virtual Camera (other hardware webcams are fine).

I tested with Google Chrome (arm64) and zoom (x86_64+rosetta2) but it did not work for me with either apps.

Logs:

2020-12-06 17:04:48.976 zoom.us[81717:3152922] Error loading /Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam: dlopen(/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam, 262): no suitable image found. Did find:
/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam: code signature in (/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)
/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam: stat() failed with errno=60

Steps to Reproduce

Trivial.

Additional information

Highly relevant to johnboiles/obs-mac-virtualcam#4. I think one needs to have the binary properly signed. I'd like to note that the obs-mac-virtualcam binary distributed by @johnboiles does not have such issues; it worked well on a M1 mac (with OBS 26.0.2).

[WizardCM]

cc @PatTheMav @gxalpha are we doing something weird?

[gxalpha]

this should have been addressed in 55da44d 🤔
especially weird since these are the same changes as on the plugin where it does work. Also while it looks similar, i dont think its the same problem as johnboiles/obs-mac-virtualcam#4, but it might be worth it trying to unsign the apps (https://github.com/johnboiles/obs-mac-virtualcam/wiki/compatibility/)
also cc @johnboiles

[WizardCM]

Then my guess is if the user updated from rc1 to rc2, the updated DAL plugin may not have been applied. Did we bump the DAL plugin version?

[gxalpha]

no, we didn't.

[WizardCM]

May be worth bumping it just because for actual release, I think. I assume if the versions don't match OBS will install the newer version?

[wookayin]

After installing via OBS 26.1 rc2:

❯❯❯ file /Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam
/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64] [arm64:Mach-O 64-bit bundle arm64]
/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam (for architecture x86_64):	Mach-O 64-bit bundle x86_64
/Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam (for architecture arm64):	Mach-O 64-bit bundle arm64

So this is indeed a fat universal binary. I think this bug is different from what has been resolved in #3779 (i.e., architecture difference).

❯❯❯ cat /Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/Info.plist | grep Version -A1
	<key>CFBundleInfoDictionaryVersion</key>
	<string>6.0</string>
--
	<key>CFBundleShortVersionString</key>
	<string>26.1.0-rc2</string>
--
	<key>CFBundleVersion</key>
	<string>26.1.0-rc2</string>
--
	<key>LSMinimumSystemVersion</key>
	<string>10.13</string>

Looks like the distributed binary has a correct version bumped.

[PatTheMav]

The plugin is also properly signed:

Executable=/Volumes/OBS-Studio 26.1.0-rc2/OBS.app/Contents/Resources/data/obs-mac-virtualcam.plugin/Contents/MacOS/obs-mac-virtualcam
Identifier=com.obsproject.obs-mac-virtualcam.dal-plugin
Format=bundle with Mach-O universal (x86_64 arm64)
CodeDirectory v=20500 size=1304 flags=0x10000(runtime) hashes=33+3 location=embedded
Signature size=8986
Authority=Developer ID Application: Wizards of OBS LLC (2MMRE5MTB8)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=6. Dec 2020 at 00:56:30
Info.plist entries=13
TeamIdentifier=2MMRE5MTB8
Runtime Version=11.0.0
Sealed Resources version=2 rules=13 files=1
Internal requirements count=1 size=204

Could you please run codesign -dvv --entitlements :- PATH_TO_CHROME_BINARY to check its entitlements? com.apple.security.cs.disable-library-validation needs to be in there for the virtual cam to work (iirc Zoom doesn't set it).

Edit I just checked, the main binary will not have that entitlement, because Chromium doesn't want it for security reasons. Instead the helpers should have it, which - in case of the Apple Silicon build - they do:

Executable=/Volumes/Google Chrome/Google Chrome.app/Contents/Frameworks/Google Chrome Framework.framework/Versions/87.0.4280.88/Helpers/Google Chrome Helper (Plugin).app/Contents/MacOS/Google Chrome Helper (Plugin)
Identifier=com.google.Chrome.helper.plugin
Format=app bundle with Mach-O universal (x86_64 arm64)
CodeDirectory v=20500 size=2283 flags=0x10a00(kill,restrict,runtime) hashes=62+5 location=embedded
Signature size=9043
Authority=Developer ID Application: Google, Inc. (EQHXZ8M8AV)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=2. Dec 2020 at 07:01:28
Info.plist entries=17
TeamIdentifier=EQHXZ8M8AV
Runtime Version=10.15.0
Sealed Resources version=2 rules=13 files=0
Internal requirements count=1 size=108
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
	<true/>
	<key>com.apple.security.cs.disable-library-validation</key>
	<true/>
</dict>
</plist>

[gxalpha]

Small part of the problem might be that the version strings are incorrectly formatted due to the -rc2-suffix.
According to the documentation they can only be [Major].[Minor].[Patch], so maybe it doesn't detect the version properly. Just a wild guess though, since the rest of the app works.
https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleshortversionstring
https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleversion
Also I'm probably not enough of a developer to be any more helpful here i think

[PatTheMav]

@gxalpha If that is the case, manually changing it in the DAL plugin's Info-plist as well as OBS' (as OBS will overwrite the plugin if the versions don't match) should be enough to test this.

[wookayin]

@gxalpha @PatTheMav Bingo! I manually modified /Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin/Contents/Info.plist such that all version strings are formatted like 26.1.0 -- while the virtualcam has been running --- and now the codesigned apps (Chrome & Zoom) can load the DAL plugin.

[gxalpha]

ok, are we somehow able to give the virtualcam a separate version number then?
While I don't like the idea of carrying around a non-standard version number, I also really don't like the idea of the virtualcam breaking during every rc-phase, and then magically working in the full release.

[PatTheMav]

So I just tested it on an Intel Macbook and rc2 works fine without issues. Which means this would only be an actual issue if macOS behaves stricter on M1 processors or maybe just notarised apps.

Edit: I'd like to get more clarity on this first, as the current process was chosen to minimise manual versioning/impact for bundling the app.

In my tests with non-notarized apps the plugin was loaded fine even with the suffix. In this report I assume we have Chrome and Zoom (both notarised I assume) which do not. @gxalpha could you test this on your machine as well please to rule out an x86/M1 difference?

[gxalpha]

On my intel mac, upgrading from rc1 to rc2 did work, with OBS asking me to give permission to overwrite the old plugin and successfully doing so. Virtual camera does still work.
Info.plist also says 26.1.0-rc2.
So in short, everything works.
I unfortunately don't have an M1 so no way to test that for me.

Edit: Also please pardon me, it's pretty late and I'm gonna sleep now, so answers might take a while

[wookayin]

In your environments, what was the macOS version? I wonder if it's a Big Sur (11.0) v.s. older OS versions, or a x86_64 vs M1 difference.

[johnboiles]

I agree with @gxalpha that it’s probably something about version numbers.

@wookayin did you have rc1 installed prior? I’ve definitely experienced issues in the past when trying to install new versions of the plug-in but I forgot to bump the version. You tried rebooting, restarting chrome, etc after installing rc2 before reporting this right? Lots of times a reboot seems to help folks and you generally need to restart host apps after changing the plug-in bundle.

[gxalpha]

In your environments, what was the macOS version? I wonder if it's a Big Sur (11.0) v.s. older OS versions, or a x86_64 vs M1 difference.

Big Sur 11.0.1, for everything that I described

[wookayin]

@wookayin did you have rc1 installed prior? I’ve definitely experienced issues in the past when trying to install new versions of the plug-in but I forgot to bump the version. You tried rebooting, restarting chrome, etc after installing rc2 before reporting this right? Lots of times a reboot seems to help folks and you generally need to restart host apps after changing the plug-in bundle.

I think your comment and my answer to that belong to #3839 (because it feels like a different issue/phenomenon), but let me write here as the discussion has moved here from #3839 :)

So yes, after rebooting, the plugin works and does not crash. Restarting the host apps without rebooting does not help. The scenario is like: (i) have a previous version working (rc1); (ii) Update OBS.app and start virtual camera to automatically install a new version of DAL plugin (rc2); (iii) at this point Quicktime, Chrome, etc. would all crash. (iv) Reboot, then it just works fine.

However, my view is that most of general users will run into exactly the same situation as they won't bother themselves rebooting right after starting the virtual camera in a new version. No warnings or messages are shown, and it isn't quite obvious for a lay-user that rebooting is required (regardless of why it happens). All other applications that use cameras will crash silently with no hint about OBS. Should we make an explicit warning or dialogue telling "reboot would be required"? Or is there any good way/workaround to make it not crash? For example, if the DAL plugin directory were completely removed prior to automatic installation of a DAL plugin, it would not crash even without rebooting.

Regarding this issue (#3838) per se --- the DAL plugin still won't work on a M1 machine regardless of rebooting, unless the version string is in the required format.

[PatTheMav]

It might be different on M1 Macs (which I do not have to confirm) but I’m updating the DAL plug-in sometimes multiple times a day without crashes or the need for reboots.

So I’m not comfortable with a blanket statement (‘reboots are needed’) as long as we haven’t confirmed the root cause.

[johnboiles]

So yes, after rebooting, the plugin works and does not crash...The scenario is like: (i) have a previous version working (rc1); (ii) Update OBS.app and start virtual camera to automatically install a new version of DAL plugin (rc2)...However, my view is that most of general users will run into exactly the same situation as they won't bother themselves rebooting right after starting the virtual camera in a new version.

I suspect this is an issue from upgrading from a non-universal plugin to a universal plugin. I bet that upgrading between two universal plugins will work just fine. And since the very first publicly released version of OBS will have a universal binary, normal users won't ever experience an upgrade from a non-universal plugin to a universal plugin and thus shouldn't need a reboot. Perhaps there's something in the OS that's caching the old non-universal version of the plugin binary until you restart -- and then it crashes when it tries to load the non-universal plugin inside an ARM process. Do you have a stack trace in the 'Crash Reports' section in Console.app you can find?

I also don't have an M1 Mac though, so I can't be that helpful in testing this hypothesis.

[wookayin]

@PatTheMav @johnboiles I forgot to mention in my previous comment that the segmentation fault bug (#3839) happened in an Intel mac (10.14). (actually on M1 mac too) For stacktrace, please refer to #3839.

I also did the exactly same test by upgrading OBS from 26.1 rc2 to 26.1-rc2-df4eb8219 (master snapshot), and this time the plugin loads without crash. I also remember I had experienced the same in an opposite scenario, i.e. upgrade from a universal binary (the third-party virtualcam plugin) to built-in 26.1 rc1 (non-universal). What you hypothesized sounds very plausible and makes sense. I now think that normal users won't experience this once we release a GA version.

[PatTheMav]

The tricky part is that the DAL plugin is loaded (and maybe even cached) by macOS/client apps themselves, not by OBS. When debugging the virtualcam plugin I had several occasions where lldb-rpc kept the plugin loaded, so deleting/overwriting it wasn't possible until I killed that process.

Quicktime was one of the candidates which somehow kept the plugin "in memory" and when I deleted it chose to crash until I rebooted (as Quicktime has Library hardening activated in Big Sur that issue is.. well.. not possible anymore).

Also Big Sur is using more extensive dyld caching than Catalina iirc (and haven't read up on the specifics yet).

[WizardCM]

Now that 26.1.0 is properly out on macOS, please update to it, reboot, and see if all issues are solved.

[wookayin]

Congrats! I have confirmed that the virtualcam plugin shipped with OBS 26.1.0 works well as expected on my M1 mac. However, rebooting my mac was necessary (before rebooting, host apps were crashing) after plugin installation and having the virtualcam started; restarting the host app only was insufficient.

It would be great to have a fix for DAL plugin versioning though, in order to make any future snapshot/RC versions working. Thanks for the support!

[WizardCM]

@wookayin Excellent, thanks for the confirmation. I've updated the ticket title to reflect the actual cause of the issue.

[johnboiles]

For me, on an Intel Big Sur Mac. I also had host apps crashing after upgrading from my plugin to the one bundled with 26.1. I removed the plugin entirely (sudo rm -rf /Library/CoreMediaIO/Plug-Ins/DAL/obs-mac-virtualcam.plugin), re-installed from OBS, then restarted host apps and everything worked again without restarting. Not sure if it will be the same on M1 Macs or not. Probably something about Big Sur's dylib caching.

[PatTheMav]

Possibly fixed by #4265.

EDIT: Using proper (persistent) build numbers is definately on my to-do list, but not part of that fix.

[wookayin]

Thanks for the fixes in #4265. However, my understanding is that the '-rc' or 'snapshot' version suffix broke the plugin, not library caching issue. I will test this on my M1 mac soon and let you know.

[jp9000]

@wookayin Did you ever test it?

[gxalpha]

Since we're now in a RC phase again, it might be worth it to test if this is fixed.

I'm not on M1 myself, but people on it should maybe try. Also sorry to everyone that just got pinged, but given that RC phases are pretty rare, testing this to me feels (at least somewhat) urgent :D

[wookayin]

@jp9000 @gxalpha I just tried out 27.0 rc1 (upgraded from 26.1.2) and the virtualcam works well on my M1.

As before, I did not manually delete the DAL plugin and installed the new one from OBS (start virtual camera). Without any further actions (e.g. rebooting or manual deleting, the virtualcam worked! Looks like #4265 did the trick. I can see the Info.plist file in the DAL plugin still has -rc1 suffix in CFBundleVersion, etc., so this appears to be not an issue anymore. FYI, the macOS version in my current environment is 11.2 which is different from when this issue was created (11.0).

[WizardCM]

Excellent, thanks for verifying. Closing this issue.