Add RFC: Create new plugin type (control), and Re-implement hotkeys as a control plugin

[cpyarger]

Summary

• create new plugin type "control"
• Reimplement hotkeys as a control plugin

Motivation

While currently we have the ability to control obs through plugins, There is no unified structure that allows for those controls to be accessed in one location, or to communicate with eachother.

This RFC is in regards to building a framework to allow for multiple control modules (eg. Hotkeys, Midi, OSC, Sound Board, Joystick, etc.)

READ RFC

[tt2468]

@Palakis has shared that he intends to work on a signal/callback api for plugins to interface with each other. This sounds like pretty much what he described.

However a unified control API might not be worth it. Take obs-websocket for example. One request usually does not just call one api function. It may first need to get an object from one api call, manipulate it, then make another call. As such I'm not sure how another api layer would make this more efficient, unless I'm understanding something incorrectly.

[cpyarger]

A good portion of the suggestion is for UI work. The plugin type control basically is adding the ability for a plugin to add itself to a single controls UI. All of the interfacing with OBS for the most part is still done using the traditional methodologies within each respective plugin.

The communications api layer is for doing what @Palakis is planning. Inter plugin communication. Preferably with a message type that is well documented and not plugin specific.

With this one should be able to say. Fire off a midi message from a keyboard shortcut, or send a websockets message from a midi button press.

I am hoping that by having these types available we can eventually add a macros plugin.
So module loads, it says hey I'm a control input passes forth the required UI data and gets added to the UI in the settings window. If it's an output plugin it passes forth a list of actions you can call. And when on an inputs mapping UI you can chose from any of the various output plugins and their actions will get added to the UI automatically. Including possible paramaters selectors. For example. Let's say I chose OBS as the output of a midi button press, I choose set mute as the action. It pulls from obs the list of volume sources and presents them in a combobox to choose from, I could also say chose osc, and have the button press send out a Boolean value via osc which I can listen for in other applications like QLC+ and change my lighting.

[cpyarger]

It's also a maintainability thing. Take a look at hotkeys. It hasn't truly changed in a long time. And people have been asking for something better. For quite a while. But hotkeys has bits in like 3-4 different files. Mixed in with a bunch of other things and not just the two hotkeys-edit by having it and any other control as a module it's easier to maintain and make changes. And usability wise. It's easier to only have to go-to one place to have to make changes to how you are controlling things.

[cpyarger]

a few more UI tweaks
Integration of the UI changes done by cg2121 in pull request2133

the obs-midi as it would be integrated.

[cpyarger]

a few people seem to be misunderstanding the scope of the project.

The primary UI - with no input type control plugins is this .

each plugin adds an entry to the list widget, and a page to the stacked widget

[cpyarger]

I have decided that leaving the backend in place for the hotkeys is a smart way to go due to how rightly integrated it is with OBS. I have pulled the GUI for it out and made some preliminary changes to support the proposed ones.

Currently I. Stuck on how to pass through the new qobject based elements through the plugin to a function is OBSBasicSettings to add a plugins new UI widget, and icon, and name, to the listwidget and stackedpages widget. My code for this test branch is located here https://github.com/cpyarger/obs-studio/tree/control-plugin-type

[cpyarger]

So, The Latest on the idea and experimental build, I am nearing completion of the "hotkeys" and "obs actions" widgets,

Each of them generate either an input or output action json string, which along with the names of the plugins from the combo boxes, makes up for 4 string values that make up a mapping.

[cpyarger]

Overall UI with controls plugins looks like this so far

with tree based dropdowns for advanced configuration windows

an example of my midi input and output widgets

and an example of an advance control page

this is how I am adding the widgets for the obs-midi example

[cpyarger]

The mapper will most likely be using QT's signals and slots for passing data between the plugins and the mapper itself.

example mapping call:

inputPlugin-Input ->emits(input action string)->mapping input broadcaster
mapper->exists(input action string)-if yes->emit (map with input overrides)
output(listening to mapper for signal)->(is plugin?)->if no, discard, if yes ->plugin execution function based on output action map and if exists input mapping "value"

[cpyarger]

rough current UI when no other control plugins exist.

[exeldro]

When no other control plugins exist you could hide the input plugin and output plugin columns to make it cleaner for users that don't need it.

[cpyarger]

Yupp! Added that function in.

[cpyarger]

So, If anyone has the time or inclination, and wants to test things out. I have the control mapper working, along with a working hotkeys 'trigger plugin', a workable framework for an OBS actions 'action plugin' with some fleshed out functions, and a Midi 'trigger plugin'.

The code can be found at the following places;

• Control Mapper OBS Branch
• OBS Actions plugin
• OBS Hotkeys plugin
• OBS Midi Plugin

[ibex-are-goats]

Thank you for doing this important work!

[cpyarger]

I have made a bunch of amendments to the RFC. and would love for further discussion on the idea.

[Fenrirthviti]

I can't speak to the actual technical proposals here, but this RFC as presented does not seem actionable in its current state. There are two RFCs being presented here, one of which is effectively a single sentence of description.

I would recommend that this be re-written and focusing on the backend changes that would be required for allowing this type of control plugin, with the UI/UX coming later, as an extension of the frontend API (websockets would be a good starting point for functionality as it might be able to be leveraged instead of the frontend API itself).

However, I can say with confidence that the proposed UI/UX will not be accepted, as it's far too complex and confusing to anyone who isn't already familiar with manually programming a midi controller interface.

We have had a lot of internal discussions on how to revamp the hotkeys interface, which I will be preparing as a separate RFC in the coming few weeks.

I'm going to enter FCP with disposition close for this specific proposal.