Implement pluggable storage

[DuncanHills]

Currently to implement a plugin, the BotPlugin class is extended and members registered with @botcmd. In turn, BotPlugin extends StorageMixin which is hard-coded to use shelve as its storage mechanism. This is very inconvenient for creating new storage implementations, as well as for testing — these dependencies should be parameterized and composed instead of inherited. This approach would allow me to easily drop in a Redis Collection as the storage mechanism, for instance, since it also supports a dict-like interface (MutableMapping). This would also easily support configurable storage backends based on a flag passed to the application.

Really any dependencies should use this method but storage is the most obvious candidate right now.

[DuncanHills]

The fact that storage is accessed by considering the plugin class itself a dictionary instead of using a dict-like member of the bot makes this not a trivial change (for seemingly no real convenience gain).

[benhoff]

I'm +1 on this issue. Making errbot more extensible would be awesome and I think mixins/multiple inheritance is a pain from a programmers perspective to keep track of.

[gbin]

I agree on this one.

I looked at it a little bit last week to see how we could make the plugin storage changeable.

We can do it either like we did for backend (special plugins) or a simple module/class reference of a dict like class in the config.

What do you think ?

[DuncanHills]

I actually started hacking on this last night. 🚨🚨🚨 Warning: tangent ahead! 🚨🚨🚨

It strikes me that plugins and their components have a lot of mutable state. It would be nice if all the dependencies (like the path for the shelve-based local storage) could be passed in upon construction of the storage backend (or other component). Similarly, config parsing should happen before plugin instantiation so the config can be passed into each plugin. When the config changes, plugins can be reloaded. This way, plugins don't carry too much state inside themselves, and also the lifecycle of plugins becomes much simpler.

[DuncanHills]

But yes I think for now

a simple module/class reference of a dict like class in the config.

and a common interface is the best approach. Simple wrappers can provide this from most key-value oriented backends.

[gbin]

Cool ascrane@, would you like to give it a shot and propose a PR for that ?

[DuncanHills]

Sure, I'm working on it now.

[gbin]

Any progress on this one ?

[gbin]

@sijis is also interested on working on this, any progress ?

[gbin]

Not much progress on this one and I think it is quite important if we want to make errbot docker friendly and HA friendly, I gonna take it.

[gbin]

done