Plugin architecture

[brianmhunt]

Having written a few plugins now for Knockout, I feel like there may be opportunities to make it easier to create, find and use plugins. Here are just a few thoughts on some of the things I feel are good practices (and clearly this is longer-term thinking in the sense that it would break the API for plugins substantially, but I thought I would write it down while it is on my mind).

The below has a lot of "ideal world", but there may be more practical or less-ambitious alternatives that are just as or better suited to the Knockout universe.

1. API ("Using plugins")

A knockout.plugins object that exposes:

1. knockout.plugins.register(plugin-spec) (called by the plugin author when the plugin is loaded)
2. knockout.plugins.use(plugin-name, options) (called by user, perhaps multiple times to change the options?)
3. knockout.plugins.load(identifier) (called by .use; may be overloaded by user)

The idea here is to streamline a consistent way for plugin authors to add make their contributions available and also for users to get started with those plugins.

In the life-cycle of a plugin, the plugin-spec could include the following parameters:

• name: The name of the plugin, as argument .use or future commands such as unregister
• handlers: An object with handlers that this plugin may add, extending ko.bindingHandlers
• provider: An object that provides a binding handler to replace the Knockout binding handler
• preprocess: A preprocessor function or list of preprocessor functions
• custom: A function that gets called to do anything not handled by the above

The idea here is to get rid of as much user-land boilerplate as possible. I.e. every plugin needn't have an addPreprocessor function, or to extend the bindingHandlers object, or have to have a custom init or setup etc function that sets the options for the plugin.

So when one wants to use knockout.punches, it would be as simple as:

// For ES6 modules/RequireJS
knockout.plugins.use('knockout.punches', {interpolationMarkup: true});
// For CommonJS/Browserify
require('knockout.punches');
knockout.plugines.use('knockout.punches', {interpolationMarkup: true});

In the ES6 modules/RequireJS world, the above would call e.g. require(['knockout.punches'], ko.plugines.register). In the Browserify world the hard-coded require('knockout.punches') needs to come before the .use call.

One would generally not want ko.applyBindings to be called while plugins are loading, and there are several ways around this, for example:

1. A plugins.on_loaded callback; or
2. A plugins.use(plugins-map) that returns a thenable e.g.

ko.plugins.use({
  'punches': {all: true},
  'else': {inlineElse: true }
}).then(function (results) {
  // results might be an object that maps return values from the `.use`
  // i.e. { punches: {}, else: {} }
  // In this way plugins could expose helpful bits of their innards.
  ko.applyBindings(...)
}, catch_err_fn)

I feel the ko.applyBindings call should for backwards compatibility always be synchronous, but perhaps it should throw an error if plugins are not loaded.

2. An online Registry

I personally like the style and setup of the Chai.js registry source control, though I would prefer the original to be in the more human-legible yaml or cson. With Chai.js plugin authors submit pull requests to add their respective plugins.

Of course, one then needs a place to find them, like Gulp.js.

In an ideal world the registry could use the Git API to check the last time the plugins were updated, the number of stars/forks, as well as any associated CI test results (which is really just guessing the url of the status image), since this is handy info.

3. Base plugin project

One of the trickiest bits with authoring a knockout plugin is getting started. It would be a great help to authors if there was a boilerplate project (e.g. knockout-plugin-base) that could be forked. It could include, for example:

1. A sensible, consistent layout (dist/, spec/, src/, README, package.json, LICENSE, etc.)
2. A configuration file (e.g. config.yaml)
3. A task runner (e.g. gulp/grunt) with the essential tasks (build, test, release, etc.)
4. A test runner that tests not only the plugin but has a spec that also runs the full Knockout test suite to see if the plugin breaks anything in the core

---

There are undoubtedly some drawbacks to the proposal above (e.g. compelling plugin authors to use one particular test runner), but I would hope that with a plugin architecture like the above the pros would substantially outweigh the cons.

This is just some food for thought, of course. It actually started out as a very short post and modest proposal, honestly. :)

[bapti]

+1 this sounds awesome!

[unsafecode]

+1

[mbest]

There are some great ideas here for encouraging the development and use of Knockout plugins. It seems to me, though, that if this system is built into Knockout, that would make it difficult for plugins to support multiple versions of Knockout, something that's usually quite important.

[brianmhunt]

Thanks @mbest. What are the issues you are thinking might come up between versions of Knockout?

There seem to be four possibilities, namely whether the Knockout version and an example plugin are respectively designed with/for the framework or not, just for clarity and illustration:

Knockout	Plugin
no	no
no	yes
yes	no
yes	yes

I think the boilerplate plugin design and the Knockout plugin framework ought to cover all these scenarios. In other words the proposed Knockout plugin framework ought to still allow current plugins to work in the same way, and as the plugin boilerplate ought to be designed in such a way that the Knockout plugin framework is not necessary (but having it reduces the amount of loading/initialization code). In other words, the plugin framework would be optional from the user perspective.

That said, do you see something intrinsic to the framework that might prevent it from working this way?

[SteveSanderson]

Thanks for raising this!

Your points #2 and #3 are very easy to agree with. I don't see any drawbacks to doing those, and it would certainly aid discoverability a lot of there were some kind of de-facto standard place to acquire plugins and extensions.

For point #1 though I admit I'm not quite as sold on what problems it would be solving. There is already a wide range of extensibility points that plugins can attach themselves to, without needing a central "register plugin" API. Having a central API comes with the impression that KO is somehow reflecting over the set of registrations and handling things like avoiding name clashes, supporting unregistrations, and so on, which would not be the case and hasn't so far been a requirement. AMD (or other module systems) already takes care of dynamic loading (and knowing when things are loaded), so for example using Knockout Punches is already as simple as referencing a .js file (or AMD module) and calling ko.punches.enableAll.

Please don't take this as a rejection of #1, since I'm totally interested - I'm just admitting that so far I'm unclear on if or how the overhead of setting up a new centralised API would pay off in terms of what pain points would be alleviated.

For #2, are you thinking we'd set up a new website, or would this somehow be bootstrapped on top of an existing package distribution system like NPM?

[brianmhunt]

Thanks @SteveSanderson -- my pleasure!

I think #2 would work on the main site, though a new site works too. It just needs to include the html and the JSON/YAML file listing all the plugins. Github pages might be adequate.

How were you thinking a bootstrapping on e.g. NPM might work?

The pain that #1 proposes to alleviate is two-fold:

1. helping plugin authors get started, and homogenizing how they do it; and
2. homogenizing how plugin users activate plugins.

Having written it out, those aren't really all that painful 😀 ... especially with #3 which could illustrate some good & strong conventions. Unless there's some pain that has been forgotten since I wrote the issue out in the first place (quite possible), maybe we can skip this for now and I'll revisit if the pain returns.

I have had a little bit of a hand in #3 by creating knockout-plugin-foundation... though I have not yet gotten very far with it.

[bapti]

@brianmhunt
Regarding the discussion around point 1 - It depends what your goal of this is. If it's to reduce the barrier to entry for plug-in authors and build a bigger ecosystem of plug-ins I would say it's the most important. Having lots of examples about how to extend the framework in a standardised way will result in much higher uptake I would think. If every plug-in registers and works differently it's just going to make it seem scary.

[brianmhunt]

@bapti Thank you so much for sharing, that's very important feedback. It is easy to lose perspective, and I hope I don't appear flip-floppy on this point. 😄

I think you have hit on the mark my original thinking on a plugin framework:— lower the barrier to entry with clearly exposed centralized API endpoints, and homogenizing for end-user consistency. Making it easier for others to make it better, so to speak.

[brianmhunt]

Here's a sample plugin website regarding point 2 above.

[brianmhunt]

Incidentally, perhaps we could pull brianmhunt/knockout-plugins-foundation into the Knockout organization - with that project serving as the foundation for points 2 and 3 above.

[jods4]

There are more than 1 issue here I think. Points 2 and 3 are good ideas. Regarding 1, I am not sure what pain point this new API addresses.

Here's my experience: I am working on medium to big projects where we create lots of components and custom binding handlers. We use Typescript, AMD modules and requirejs (with optimized production code using r.js).

Declaring a new component or handler is very easy and done in a single file. It contains the code plus the registration line (so those things all require('ko') at the top).

Using it is just a matter of making sure they have been loaded. Inside the viewmodels that need a custom handler, we put <amd-dependency path="Bindings/Whatever" /> at the top, which is just TypeScript way of requiring something you don't directly use. With plain javascript that would just be a plain require() dependency.

In the end I don't see how additional APIs would make that easier. Maybe for other kind of extensions such as preprocessors (which admittedly we don't use much)?

[brianmhunt]

@jods4 - thanks for the feedback. You're right that for userland stuff there is not much pain here.

I was thinking it might make it easier for plugins like knockout-else, knockout-punches, knockout-secure-binding, and others to register things like preprocessors. Right now there is a lot of duplication of code - sometimes a big percent of a small plugin - just to avoid clobbering other plugins.

While the number of plugins / plugin authors is relatively much smaller than users, the idea is to lower the barrier to entry for those who want to push out what might be a useful plugin for others to use.

[brianmhunt]

A good example of what I was thinking for the plugins architecture is not hugely ambitious, being along the lines of the 135 line microplugin.js library.

[jods4]

By some strange coincidence I recently had to create some binding preprocessors. Now I see what your first point is about.

If you want to write a custom binding handler with its own preprocessor it's easy enough. But if you want to hook your preprocessor into other bindings (e.g. the built-in ones), then the problem is that the preprocessor API only supports one preprocessor per binding as the ko.bindingHandlers.<name>.preprocess function. This makes it very awkward because you don't want to erase other preprocessors that might already be there: you usually want to chain them.

Isn't the solution to this problem to simply build @mbest's helpers (found in ko.punches.utils) into KO proper? So plugins can just do ko.addBindingPreprocessor(...) rather than re-implementing this chaining logic themselves?

[brianmhunt]

@jods4 Yes exactly— adding @mbest's preprocessor in is the basic idea, though it would be good to have either or both of 1. dependency management, and 2. ordering of preprocessors. Which is what the microplugin library does.

[jods4]

I can't help but feel that dependency management is going a little overboard here...

If you have preprocessors that depend on one another (is that common? not in my experience), or maybe on a specific binding handler, it's a matter of declaring them in the right order in your page or bundling tool -- or better yet use an AMD system such as requirejs and declare your dependencies properly (that's what I do).

I think we're far from the TextEditor with open plugin architecture that is the example given in microplugin readme.

[brianmhunt]

@jods4 Thanks. The dependency management is rather simple and straightforward, but opens a whole new class of dependency chaining that some argue is otherwise impractical. It's worth at least considering. :) The trick/benefit is for it to remain build-system agnostic, as some folks are using CommonJS, AMD/RequireJS, and soon there'll be ES6.