Add RFC: Virtual Camera #15
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| - Start Date: 2020-03-25 | ||
| - RFC PR: #15 | ||
| - Related Github Issue: https://github.com/obsproject/obs-studio/issues/25685 | ||
|
|
||
| # Summary | ||
|
|
||
| Add the ability to render output to a virtual camera device on Linux, Mac, and Windows. | ||
|
|
||
| # Motivation | ||
|
|
||
| OBS is a powerful set of tools to manipulate live video streams that natively supports output to popular streaming services as well as rendering to a local video file. There are a huge number of people who engage in 1:1 or small-scale streaming using video conferencing software like Zoom or Google Hangouts. Many of these people have similar needs to those of traditional streamers, but for one reason or another cannot switch the video conferencing software they use (social graph, corporate policy). | ||
|
|
||
| OBS can meet this need by creating a virtual camera device and outputting to that device such that programs capable of consuming camera input can now consume OBS input with no extra development. | ||
|
|
||
| Similar functionality: | ||
| * OBS can be extended by the [OBS-VirtualCam plugin](https://obsproject.com/forum/resources/obs-virtualcam.539/) (Windows) and [obs-v4l2sink plugin](https://github.com/CatxFish/obs-v4l2sink) (Linux). Currently, there is no OBS solution for macOS. | ||
| * [Wirecast includes virtual camera support](http://www.telestream.net/pdfs/user-guides/Wirecast-8-User-Guide-Windows.pdf) on both Windows and Mac. | ||
| * [Webcamoid](https://webcamoid.github.io/) is an existing open-source cross-platform virtual camera application, which may have some useful tidbits when investigating implementation details for each platform. | ||
|
|
||
| # Detailed design | ||
|
|
||
| Feature Set: | ||
|
|
||
| * A button is added to the Output Controls dock labeled "Start Virtual Camera", which will toggle the start/stop status of the virtual camera output. While virtual camera output is active, the label should read 'Stop Virtual Camera". | ||
| * The virtual camera appears in the system listed as "OBS Virtual Camera". | ||
| * The virtual camera outputs at the same resolution and framerate as the OBS rendered output. | ||
| * A section is added to OBS settings providing the following settings: | ||
| * A checkbox to indicate if the user wishes to start the virtual camera output automatically when OBS starts | ||
| * A checkbox to flip the virtual output horizontally | ||
| * In the Output settings tab, the warning that appears when outputs are active is augmented to list all active outputs so that users know which outputs they need to disable in order to edit output settings. | ||
| * When the virtual camera output is not active, the virtual camera device shows a static image to consumers indicating to the consumer that output is not started. This image meets four requirements: | ||
| * It allows the consumer to smoothly switch from non-active to active content without having to worry about what order the applications are started in. | ||
| * It informs the user that the output is not started yet, and needs to be started before output will be sent from OBS. | ||
| * It is aesthetically pleasing such that, if the output is accidentally sent to viewers (for example, if the user fails to start the virtual output before joining a Zoom call), it is not unnecessarily gaudy or technical. | ||
| * It requires minimal localization, or provides a way to localize any displayed text on the image | ||
| * The auto-config tool is updated to include "outputting to a virtual camera" as their primary use case, with reasonable recommendations. Since this doesn't require testing encoding or bandwidth settings, the resolution should be set to 1920x1080 and frame rate set to 30FPS. | ||
| * A new command line flag `--startvirtualcam` is added, which starts the virtual cam automatically when OBS is started. | ||
|
|
||
| ## Platform specific implementations | ||
|
|
||
| * On **Windows**, the implementation of this plugin should leverage [libdshowcapture](https://github.com/obsproject/libdshowcapture). OBS already depends on libdshowcapture, so using this existing library service to limit the number of extra dependencies that OBS needs | ||
| * On **macOS**, OBS will need a plugin to output over IPC to a CoreMediaIO DAL plugin that is registered on the system upon OBS install. This will require repackaging the installer as a `.pkg` instead of a compressed `.app`. | ||
| * On **Linux**, the most straightforward way to implement this functionality would be to output via [v4l2sink](https://gstreamer.freedesktop.org/documentation/video4linux2/v4l2sink.html?gi-language=c). Note that this still requires the user to have [v4l2loopback](https://github.com/umlaeute/v4l2loopback) installed to consume this output. For package installations, a dependency should be placed on v4l2loopback such that it is installed by the package manager when OBS is installed. | ||
|
|
||
|
There was a problem hiding this comment. Inserting quote from Jim:
In summary:
|
||
| ## Technical considerations | ||
|
|
||
This comment was marked as resolved.
Sorry, something went wrong. |
||
| - This RFC is explicitly for an OBS-specific implementation of output from OBS to a virtual camera. A generic middleware solution to this problem is out of scope for this problem. | ||
| - The virtual camera will need to be registered with the system upon OBS installation. | ||
| - The virtual camera should be accessible by third party applications that support webcams/capture cards | ||
| - OBS supports running multiple instances of itself. If two instances of OBS run simultaneously, the first instance to output to the device should "win". The second instance should give an error saying that the output device is already in use by another instance of OBS. | ||
|
|
||
| ## User UX | ||
|
|
||
|
There was a problem hiding this comment. Initial configuration should require very little input from the user, if any. Resolution and framerate of the camera will be pre-defined by the Video settings of OBS, and no effects outside those already provided by OBS will be specially exposed for this output. There was a problem hiding this comment. One of the things I don't like about the obs-virtualcam plugin is that, in the installation, it asks how many virtual cameras you want to create. While this is a neat feature for someone who understands what they are doing, I think it's too complicated for generalized OBS use. I think it's perfectly acceptable to assume a single camera output from OBS that outputs whatever is going to PGM. The only setting I really could see being useful would be a checkbox to auto-start the output on OBS launch (though even that might be bad, since having an active output can confusingly lock output settings). |
||
| * When installing, OBS should OBS should install the platform specific driver to enable virtual camera output on the system | ||
| * Initial configuration should require very little input from the user, if any. Resolution and framerate of the camera will be pre-defined by the Video settings of OBS, and no effects outside those already provided by OBS will be specially exposed for this output. | ||
| * Add a button in the OBS UI below "Start Recording" and above "Studio Mode". In English localization, the button would be labeled "Start Virtual Camera". | ||
| * When you click this button, the current output is directed to the virtual camera device. The button text then toggles to the localized "Stop Virtual Camera" label. | ||
| * When the user clicks into the settings of their video conferencing app, they will see an entry in the list of cameras labeled "OBS Virtual Camera" (no localization). When they choose this, the OBS output will be fed from this "device" to the app. | ||
| * When the user clicks "Stop Virtual Camera", the virtual device no longer receives frames from OBS, and instead receives a static image defined by OBS indicating that the output is not active. | ||
|
|
||
| # Drawbacks | ||
|
|
||
| * The most obvious omission from this RFC is the lack of virtual audio output in addition to virtual camera output. However, though the features are logically closely related, they are unfortunately vastly different technically speaking. As such, we will leave the implementation of virtual audio output to [a separate RFC](https://github.com/obsproject/rfcs/pull/16). | ||
| * Platform-specific driver code would require maintenance as operating systems (especially in more recent times) tend to limit access (or require user permission) to access/provide to AV hardware. This especially will mean more testing/debugging will be required each time Windows, macOS, and officially supported Linux flavours (currently Ubuntu) provide a major update. | ||
|
|
||
| # Alternatives | ||
|
|
||
| * Pursue a more generic output interface and (encourage someone else to?) build a generic virtual camera to consume it. E.g. output via RTSP, NDI and have a "NDI Cam" that has nothing to do with OBS. | ||
|
There was a problem hiding this comment. Note: all these alternatives require more effort for the user to know what needs to be done. Keep in mind, the goal is to make it easy for anyone to install OBS, and start using a virtual camera. There was a problem hiding this comment. I wouldn’t consider NDI viable: it’s very very picky to get working reliably for long streams, and older hardware (e.g first gen trash can macs) have trouble with high resolution FPS. On long streams, AV sync isn’t an issue I’ve been able to resolve fully. Additionally, if built in, licenses may be an issue with NDI’s libraries |
||
| * Adapt Webcamoid's source code for our use. This may end up being more involved than we'd like, however, and might end up making the feature more complicated than we want. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As an alternative to this, when the user first starts the virtual camera, we could potentially prompt to install the DAL plugin at that time. We'd need to show a password prompt to be able to have access to write to
/Library.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How is that better than installing it at OBS install time? It's already elevated at that point and cuts out an extra step.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On macOS, since OBS distributes a
.appbundle inside a.dmgimage, we don't have elevated privileges at install time currently. When the user first opens OBS, macOS prompts for permission for things that require elevated approval/privileges (e.g. Microphone, Camera, etc). A common pattern I've seen recently for apps that need additional files installed is that they distribute as a.appbut have menu actions to install things with elevated privileges. This allows the end user to decide to what extent they want to give an app access to their system.Example from JetBrains' IDEs
Example from VSCode
For me, I'm generally suspicious when apps require a
.pkginstaller since it's harder to know exactly what they're installing on my system. I try not to install apps via.pkgif I can help it..pkginstallers can install sneaky things that break security like Zoom did. Maybe others feel similar to me, or maybe they don't care 🤷But it probably doesn't really matter either way. I bet OBS has enough notoriety that it won't really affect how people perceive it. And if we believe that the virtual camera is going to be used by the majority of OBS users, then a
.pkgis definitely the right way to go.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO this is absolutely miserable UX. I'm fairly confident that the wide majority of people will absolutely not care if it's a
.pkgor an.app, but they will care if it works out of the box or not.The Mac build of OBS was distributed as a
.pkgfor years with no objections, and it was only switched to a.appwith version 24.0.0 because it didn't need to be a.pkg. Shipping a.pkgwould just be going back to the way things used to be, except now it's properly justified.Finally, unlike Zoom, OBS is an open source application that anybody can inspect, down to the packaging scripts. If people were concerned, they can inspect the source code and the packaging scripts themselves.
I appreciate the concern on this issue, but as a matter or practicality I don't think we will have anything to worry about vis-à-vis public perception.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
100% agree. But it's miserable UX in the name of security, which is basically the whole story of macOS Catalina 😂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue there is that we don't really have much onboarding to speak of for Mac, other than the one-time prompt to run the auto-config, so that would have to be developed almost from scratch.
For what it's worth, we really should already have an onboarding flow for Mac to do things like request permission to capture the microphone/screen/camera.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, what I tried to get at was that the menu bar item would be needed anyway and thus would still be a necessary implementation detail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, fair point then. Do you think an entry in the Tools menu is appropriate for this? Or do you have any other ideas of where it would best go?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah "Tools" seems to be the current home for all additional functionality and plugins. Simplest implementation might be to have the current single item do the additional checks and installation of the DAL plugin and keeps the "Enable" state if it fails or a user cancels the process.
There might be more ways to make this menu item more verbose about state of the plugin, but I have no idea if it's feasible using Qt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@johnboiles I've been working on code for an onboarding flow and also investigated how to install/update the DAL plugin at runtime without the need for an installer with elevated privileges.
The short story is that it's possible with a little "hack" as OBS is not a sandboxed app and thus the usual ways to get elevated write permissions for the target folders do not work (e.g. show a file browsing dialog defaulting to the target dir, which would give write permissions for a file operations).
The only way to make this work is to invoke an AppleScript that runs a
cp -Rwith elevated privileges. It does work without issue though.Now the pickle: We could have this code run as part of the plugin when it's being activated, then remove it later once we have a proper macOS onboarding flow. It would allow release of this plugin earlier, but introduces a small amount of maintenance later.
This in essence does the deed:
https://github.com/PatTheMav/obs-studio/blob/bd849551e6dcd8c525e60e35d1f5d046a39ca53c/UI/platform-osx.mm#L153
When OBS is run as a bundled app, the permission dialog will look "proper", alas there is no way to add an explanation what OBS needs it for, but I hope that triggering this through the menu item will make it clear (excuse the German-ness of it):
The DAL plugin is copied dutifully to where it should be and I checked that e.g. Quicktime picked up the virtual cam output directly after copying over the plugin (no restarts of OBS or the system needed).