Plugin architecture

[pipermerriam]

What is the big idea!?

I have been thinking about a new API for plugins in the trinity client.

Plugins could range from things like simple internal optional components such as a gas estimation engine up to full blown applications like daemon processes for something like augur or ethereum alarm clock.

I also think that we might benefit from dog-fooding this API for many of our internal things such as the JSON-RPC server, the transaction pool, peer management for rating and blocking peers.

Here is a quick list of things that I think fit well into a plugin type architecture.

1. Application which manages paid slots for light peers
2. Transaction Pool
3. Rating peers
4. Blocking peers
5. Whisper
6. Deamon processes (augur, ethereum alarm clock, truebit)
7. The python console.

APIs that would need to be exposed for these things would be:

• hooks to add new sections of CLI arguments
• hooks to add new CLI commands.
• high level Web3.py connection
• low level chain and database access
• hooks for interacting with core components like peer pool
• hooks for launching JSON-RPC server namespaces.

We might even be able to go as far as proving the ability to run new DevP2P services which would allow running something like whisper.

And finally, while this is the absolute extreme version of this idea, we may even be able to move syncing to be relegated to run within this system.

How can it be fixed

This probably starts with a POC for either the transaction pool or the planned system for rating and persisting information about peers.

[cburgdorf]

There also needs to be a generic way for plugins to raise events that other parts of the system (either core stuff or other plugins) can react upon.

One idea could be to have a generic event bus in Trinity that is exposed to the plugins and where plugins can listen for events but can also raise events. An event bus has been my standard pattern for this kind of problem in the past (which is why I'm bringing it up here) but there may be other well suited patterns as well.

[cburgdorf]

I just developed some ideas for this in my head. I'd like the TxPool to be written as a plugin that also defines a --no-tx-pool cli arg so that we have a way to easily disable it (e.g. it's buggy).

So, my first use case for the plugin architecture would be to:

1. Don't hardcode the TxPool in the core trinity code base
2. Furthermore, avoid hardcoding a TxPool specific cli arg into the core Trinity code base

I also have some ideas on how to tackle inter plugin dependencies. Some of this stuff reminds me of how we handle the validation pipeline at the MachineLabs server.

I'd like to take a stab on this 🔪

[cburgdorf]

Just leaving this thought here: I think that the nodes itself (full (ropsten/mainnet), light (ropsten / mainnet)) are pretty good candidates to fall into the plugin space themselves. We can probably make these things more fine grained and then provide plugins to expose the same functionality as today but with the added benefit to make it really easy for people to e.g. add support for Kovan or whatever.

[cburgdorf]

Guess we can close. Feel free to re-open if anyone disagrees.