Figure out a way to load arbitrary plugins at runtime

[nukeop]

Webpack doesn't make it easy to load js from regular text files at runtime, and I'd like to make it possible for savvy users to extend Nuclear's functionality with any code they want. For example, the plugins section in the left-hand menu could include a button that would let the user open a js file with a plugin (the api for plugins should be offer the ability to change anything in the program but how it's going to be structured is a discussion for another day).

It could be worth investigating how ava does this (because it has to load tests at runtime if I understand it correctly).

Another option could be letting plugins interact with the http api, this would open up the possibility for plugins to be written in any language, but would require including libraries for making http requests, and a mechanism to integrate them with Nuclear.

[Venryx]

I definitely think there should at least be an option for direct Javascript plugins. This is because, while an HTTP API is great to have, there will always be a gap between the features present in the player and the features exposed through the API. By allowing for direct Javascript plugins, you let developers "plug gaps" in functionality without having to extend the HTTP API for each one.

As for how to structure it, I assume the difficulty you're referring to is how to "expose" the functionality of the program to the loaded JS files. A "hacky" way to do this is just to expose the master "modules" collection of the webpack system. An example of this approach can be seen here: https://github.com/Venryx/webpack-runtime-require

The JS plugin file can then look through that for the functions and objects it needs, and interact with them. This is of course pretty fragile, but perhaps is worth adding as a low-level fallback for personal plugins where the user is not too concerned about long-term robustness, and just wants to add a quick modification to something.

For the long-term question of how to structure the plugin access/contributions/modifications, I'm not sure; I haven't implemented an extensive plugin/hook system before, so would have to see how other projects do it before I felt confident enough to make real suggestions.

[nukeop]

The easiest way would be to put some kind of an api in the global scope and just eval loaded code at runtime. I might try to do a POC soon...

I'd like each plugin to contain some metadata like: name, description, icon, etc. This way we can think about using npm as a repository for plugins and include downloading them in the program.

[Venryx]

If you implement a proof-of-concept plugin system, I'll likely start using it right away.

Up until a few days ago I was using Clementine as my music player, but it's written in C++ so it's too much work to implement the various features I'd like to see. Since nuclear is written is Javascript (my primary language), I'm much more comfortable extending it.

Long-term I plan to submit improvements to the main program, but for now I'm just looking for a quick way to add in some changes that will make nuclear well-featured enough that I can start using it as my main player.

Main things I'm looking to add:

1. Better local-library management (eg. collecting songs within subfolders -- for me, nothing happens if I just select my root library folder)
2. Smart playlists. (eg. collecting songs with X+ stars from throughout my library)
3. ReplayGain support.
4. Controls to view and modify track star-count.
5. A "weighted shuffle" system, letting me have higher rated songs play more frequently, as well as manual controls to change the weighting/play-frequency-multiplier for a given folder or album/song.

I can live without 3, 4, and 5, but the first two are ones I'll need to implement before I switch over to nuclear completely. I'll start the work within plugin/eval code, but later on (after familiarizing myself with the program structure) I'll hopefully have the time/motivation to bring those changes to the main program (in the form of pull requests).

Regarding the distribution approach for plugins, npm sounds like a reasonable approach. It's free, people are familiar with it, and it's light-weight (basically just a package.json file), so less concern about compatibility getting broken down the road.

[nukeop]

A small update on this, I'm trying to handle this in an easy and dirty way by loading js and running eval on it. Maybe later I'll run babel on the loaded scripts before running them. There will be an api object passed to the script's start function, and the script will have to define some keys like name, description, etc. In the start function, you'll get easy access to some features, and you'll be free to run any js you want. Maybe in the future I'll also add some callbacks to be called when certain things happen.

[nukeop]

I've committed some code here: https://github.com/nukeop/nuclear/commits/feature/user-plugins

If there are any suggestions on what to add to the plugin API, I'm open.

[Venryx]

Fantastic! I'm not yet familiar enough with the source to have suggestions on things to add to the api, but once it's in the public release, I plan to "officially" make the switch from Clementine and trying to monkey-patch in support for the "must have" features on my list.

At that point, it should become clear what sort of API hooks would be nice to have. (as opposed to the general api.getCurrentWebContents().executeJavaScript(code) approach, which may have difficulty overriding certain functionality)

If it's possible to monkey-patch in support for those features, I'll go with that at first and then, once I've gotten it polished some, create a pull request for those features (in order of simplicity, to learn the project standards along the way). If not, I guess I'll have to try a more involved development approach where I get the project building locally before implementing initial feature support.

(The reason I prefer the plugin-first development approach is that, there will always be some features I want that I know are specific enough to me to not be worth trying to add to the main program, and it's nice to be able to implement all manner of features desired in the same place -- where I don't need to worry about polishedness of code or level of usefulness to the general user at first.)

[Venryx]

By the way, is there any way to enable the Developer Tools in the public release? It would be very helpful when trying to debug plugins, but the regular ctrl+shift+i hotkey doesn't work.

Perhaps there is a command-line flag that can be passed?

[nukeop]

There is no way yet but I will add a shortcut and a button along with the user plugins feature.

[nukeop]

So for now the api will provide access to app (from Electron), the redux store, as well as React and ReactDOM. This will allow plugin developers to dispatch any actions to control the behavior of the program, as well as inject nodes into the document tree to render custom ui. Can't expose the entire Electron API, because, for reasons unknown, this causes the entire application to freeze as soon as a plugin is loaded.

[nukeop]

Ok, this is now on master.
For details, see the docs: https://nuclearmusic.readthedocs.io/en/latest/plugins/

Let me know if anything's unclear. Keep in mind that right now the plugins aren't transformed by babel, so you can't use es6, etc, unless you transform them yourself first. I will work on that feature next though.

[Venryx]

That's awesome!

The next time I have a good block of time available, I'll fire up vscode and start working on those "core features" that I need. (I'll be writing it in TypeScript with Webpack, since that's what I'm used to, and transpiling it down to es5.)

Once I've worked out a baseline implementation for those features within the plugin, I'll clean them up some more, then look to add them to the actual program through pull requests.

Exciting times, being able to use Javascript to augment my (soon to be) primary music player!

[nukeop]

I also added babel transpilation to the stack, so you don't need webpack anymore. I guess for typescript you still do. But if there's enough demand, I might add built-in transpilation too.