Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RFC: Plugin Manager #4

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 114 additions & 0 deletions accepted/0003-plugin-manager.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
- Start Date: 2019-12-30
- RFC PR: #4
- Mantis Issue: N/A

# Summary
Create a plugin manager that is capable of configuring the loading of plugins as
well as providing an easier means of installing plugins. Perhaps - later down
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A first version should not just be capable of installing - but also updating. That's one of the biggest things missing.

the line - creating a online database that individuals can search and seamlessly
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of this RFC, I'd like us at least determine whether the Resources plugin we're using on the forums fulfils our needs, or whether we will need to migrate to something with a better API, or whether we need to look into writing our own.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have absolutely no doubt that we will need to move to a different API, likely a custom web application. I envision something similar to the Kodi add-on system: https://kodi.tv/addons

install plugins.

# Motivation
Varying from plugin to plugin, there are a variety of different intallation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While these are valid motivations, there are actually two more important motivatations:

  • Discoverability
  • Compatibility (OBS updates vs plugin updates)

Many users don't know plugins exist. If they do manage to find them, they may struggle to install them. Once installed, users are never notified of updates. These often cause compatibility issues down the line.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Architecturally a plugin manager is necessary to finally differentiate between our runtime modules and actual plugins. Right now the program architecture doesn't differentiate between them and loads any 3rd party plugin as if it were a 1st party runtime dependency.

Finally making a distinction between them (which would be necessary to make a plugin manager work) would decouple both types of runtime libraries and increase software stability:

  • OBS Studio could be made to skip loading 3rd party libraries entirely (safe mode)
  • OBS Studio could be made to migrate 3rd party libraries to a different API that exposes much less internal data, enabling more safety and stability

So on top of the obvious wins for end users (discoverability, manageability) there are maintenance and architecture wins for the program itself.

methods for each plugin. On top of that, once installed, there is no easy way to
disable, or even configure, each individual plugin. In addition, if one plugin
doesn't behave correctly because of another, it can become difficult to diagnose
which plugins are causing the error. By implementing a plugin manager, it allows
users to seamlessly install plugins and have finer control over the loading of
their plugins. Furthermore, having a plugin manager opens up many more
possibilties for both users and developers.

# Design
## Functionality
### Loading
Currently, when plugins cause issues, the only way to disable them is by
outright uninstalling them. However, this could be mitigated at startup by
having an internal record of which plugins are installed. By keeping a list -
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A plugin blacklist (rather than whitelist) is a good idea. On top of this, having a way to disable all third party plugins (ala Safe Mode) to easily confirm whether an issue is caused by a plugin would also be useful.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just saw this comment and thought I should mention, the open source software community is moving away from using the terms "blacklist/whitelist" and adopting "blocklist/allowlist" or other representation to avoid certain connotations. Probably a good idea to follow suit.

perhaps in the configuration file - of the disabled plugins, they could be
ignored during initilizaiton.

Furthermore, if a plugin causes a major failure while loading or doing use, it
Copy link

@billyjbryant billyjbryant Jan 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding in addition to this, a performance measurement of which plugins cause the highest hits to system performance would be beneficial. This would help with diagnosing issues that are being caused by bad plugins not properly managing their resource utilization. Also knowing which plugins take the longest to load on OBS launch may become handy as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OBS module load time is already logged in the OBS logs at the end of each OBS session under "obs_load_all_modules" in the profiler.

will be automatically disabled on the next startup with a popup warning the
user. The popup will allow the user to re-enable the plugin (perhaps they
knowingly used a experimental feature, etc.) or keep the plugin disabled. It
would also be helpful to show an error code to the user to help them and the
developer debug.

### Installation
To further ease the user with their plugin (once downloaded), installation could be handled in 3
different ways.
- A file explorer in the manager to install a compressed package that contains:
a manifest, locales, the actual plugin, and other needed components.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that OBS already has some level of split for these:

  • Locales and resources are stored in data\obs-plugins\<plugin name>
  • Dlls, PDBs, and core files are stored in obs-plugins\<bitness>\

The biggest concern I have is that while they're split nicely in data, obs-plugins is not given the same level of care. This makes uninstalling plugins basically an unknown. I would say plugins should be given two path variables, config/resources and core, and then can define subdirectories within them. When we install, we generate a subdirectory for the plugin, and never let them work outside of it.

Additionally, I really really don't like that third party plugins are installed in the application's directory, or at least that they're mixed with included plugins. I would vote we do a similar thing to themes. Themes can read from a themes subdirectory within the config directory, which has the added benefit of a full uninstall/reinstall retaining all installed themes/plugins.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since that comment we have obviously merged lots of changes on obs-studio, including the bundle-based plugin format on macOS, which bundles (hence the name) everything a plugin consists of into a single package (albeit it just being a directory on macOS, it is treated as a self-contained package by the OS), which holds all binaries, support files, and resources that make up a plugin.

Removing a plugin just requires removing the package (whether manually or by a plugin manager). A similar pattern would probably benefit Windows and Linux users as well.

- A file explorer that installs plugins by themselves - mostly for compatibilty
with previous plugins
- A compressed package that is associated with OBS so when double clicked, it
can be automatically installed in the correct locations.

If the user installs a compressed package, the package would be decompressed and
the contents would be copied to a directory in the plugins folder.

### Manifest
As outlined in installed, a compressed package will contain multiple components
including a manifest. The manifest will be responsible for outlining all
important information about the plugin. An example of a barebones manifest is
shown below:

```json
{
"name": "Plugin Name",
"author": "Author Name",
"description": "Description that would be displayed to user",
"locale": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As outlined above, plugins would preferably have a "self-contained" bundle structure, and the same should apply for the downloaded package, which in a best case scenario would contain:

  • Windows x64
  • Windows ARM64
  • macOS Universal
  • Linux x86_64
  • Linux aarch64

(I included arm64-based architectures on purpose because those are becoming more popular thanks to Apple Silicon and it would be beneficial if we mind such future support.)

Usually the plugin package should have a fixed expected directory structure, so the manifest just states which platforms are supported and the plugin manager checks for the specified directories (i.e., if "Windows x64" is supported, a directory "windows-x64" needs to exist in the package).

Each subdirectory then contains the expected bundle with an expected filename (this could be the value of a filename field in the manifest) - on macOS we already have the .plugin bundles, a directory that would need to be copied verbatim, I leave it to Windows and Linux maintainers to decide on their preferred bundle naming scheme.

"en": "file path"
},
"plugin": {
"64": "file path",
"32": "file path"
},
"categories": ["PRODUCTIVITY", "CLASSIC", "THEMES"], //etc
"version": "^24.0.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would allow the UI to show a warning if a version doesn't match, or in case it's an outdated download, very useful. More discussion on the formatting of the version would be good, as I'd like devs to be able to specify a range. I think npm's semver matching works really well for this.

}
```

The purpose of the metadata in the manifest is for 3 main reasons: 1) Allowing
for easier parsing of the compressed package, 2) allowing more flexibility for
plugin creators, and 3) reducing mental overhead for users organizing their
plugins.

Some notable entries on the manifest are categores and version. Version
numbering allows the plugin creator to specifiy a specific version or range that
the plugin can be loaded in. The categories affects the UX.

Note: The options in the manifest are predefined. A plugin developer can not
have random options that can be searched as outlined in UX.

### UX
To access the plugin manager, it would be under Settings -> Plugins or File ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fear this wouldn't be discoverable enough. Maybe a quick-link from the main window's toolbar, too?

Plugins. The plugin manager itself would have a searchbar at the top that allows
searching through the metadata as provided in the manifest file and displays the
results in the list below. Moreover, there would be a simple list view that
outlines all the installed plugins in alphabetical order. The list would show
the name, description, and categories of each plugin. All the way to the right,
would be a button that outlines whether the plugin is disabled or currently
active. Another feature would to allow the plugin to have its own settings menu
which would display a button next to the disable button that opens its settings menu.

When a user disables a plugin, it will add the plugin to a list to prevent
loading during initilization. It will also attempt to unlink the plugin, but a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the plugins backend is not designed for dynamic unloading, and it may be dangerous to assume plugins will always fully clean up after themselves. It is most likely that the first version will require a restart to unload.

restart might be necessary. Likewise, when a user re-enables or installs a
plugin, it will attempt to initialize the plugin.

If the plugin has its own settings menu, the information could be stored in its
manifest which would be unpacked and installed.

### Future Considerations
It would also help plugins be discovered if there was a built in browser for
searching through plugins on a server. A user could search by name or other
metadata (again outlined in the manifest) which would be in another list. Then,
they could click on an install button next to the options to automatically
download and install the plugin.

# How We Teach This
It would be provided in the documentation. However, the bare functionality would
hopefully be designed well enough that it can become plainly clear to the user
how it works.