New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pylightning: Add a plugin framework #2161

Merged
merged 6 commits into from Dec 15, 2018

Conversation

Projects
None yet
3 participants
@cdecker
Copy link
Member

cdecker commented Dec 10, 2018

This framework is very much inspired by flask, in that I wanted to make it as trivial as possible to get started. I had to jump through some hoops but I'm rather happy with the end result. A plugin with an option and a RPC passthrough is as simple as this:

from lightning import Plugin, monkey_patch
plugin = Plugin(autopatch=True)

@plugin.method("hello")
def hello(name, plugin):
    """Say hello"""
    greeting = plugin.get_option('greeting')
    plugin.log('{} {}'.format(greeting, name))
    return s

plugin.add_option('greeting', 'Hello', 'The greeting I should use.')
plugin.run()

It has support for monkey-patching stdout and stderr by default so that if run by lightningd it'll automatically wrap anything written to those streams into log notifications that get displayed nicely.

Depends-On: #2154

@cdecker cdecker added the plugin label Dec 10, 2018

@cdecker cdecker self-assigned this Dec 10, 2018

@cdecker cdecker requested review from rustyrussell and renepickhardt Dec 10, 2018

@cdecker cdecker force-pushed the plugin-6 branch from b694e5e to 792964c Dec 10, 2018

try:
result = {
"jsonrpc": "2.0",
"result": self._dispatch(request),

This comment has been minimized.

@ZmnSCPxj

ZmnSCPxj Dec 11, 2018

Collaborator

It looks to me that this will result in "linear" dispatching, i.e. commands are handled one-at-a-time.

Would it be possible to have this handle dispatching in parallel, i.e. if a command stalls processing, another command can be processed?

Or is some Python limitation exists to prevent this, or it becomes complicated?

(maybe there can be a separate decoration for methods that might block)

This comment has been minimized.

@cdecker

cdecker Dec 11, 2018

Member

Good point, hadn't thought about it yet. I wanted to keep away from multi-threading as much as possible, but I'll try to see what I can come up with.

This comment has been minimized.

@cdecker

cdecker Dec 11, 2018

Member

I opened issue #2167 to track progress on this. Would love to hear feedback for the initial idea, which is future-inspired, but since the python futures are tightly bound to multi-threading I'm thinking of implementing my own AsyncResult instead.

This comment has been minimized.

@ZmnSCPxj

ZmnSCPxj Dec 11, 2018

Collaborator

This seems sensible.

@@ -895,6 +895,7 @@ void plugins_init(struct plugins *plugins, const char *dev_plugin_debug)
plugins->pending_manifests = 0;
uintmap_init(&plugins->pending_requests);

setenv("LIGHTNINGD_PLUGIN", "1", 1);

This comment has been minimized.

@rustyrussell

rustyrussell Dec 14, 2018

Contributor

Convenient! We could have also added a commandline option, but this is easier for plugins to grab in most languages.

and stdin filedescriptor, so if we use them in some other way
(printing, logging, ...) we're breaking our communication
channel. This function
"""

This comment has been minimized.

@rustyrussell

rustyrussell Dec 14, 2018

Contributor

Did the rest of this comment get redirected to JSON? :)

This comment has been minimized.

@ZmnSCPxj

ZmnSCPxj Dec 14, 2018

Collaborator

It is a common problem that occurs when monkey patching system internals. The comment

This comment has been minimized.

@cdecker
@@ -17,6 +17,7 @@ class Plugin(object):
def __init__(self, stdout=None, stdin=None, autopatch=True):
self.methods = {}
self.options = {}
self.option_values = {}

This comment has been minimized.

@rustyrussell

rustyrussell Dec 14, 2018

Contributor

I don't understand the distinction between options and option_values?

This comment has been minimized.

@cdecker

cdecker Dec 14, 2018

Member

options are the names of options that we registered with all their metadata, whereas option_values are the ones that were set by the init call from the plugin. We can merge them if desired of course.

@cdecker

This comment has been minimized.

Copy link
Member

cdecker commented Dec 14, 2018

Merged options and option_values as suggested by @rustyrussell and fixed the comm

cdecker added some commits Dec 10, 2018

plugin: Set LIGHTNINGD_PLUGIN env var inform plugins
It might be useful to take special precautions inside a plugin when
being run as a plugin (and not as a standalone executable). This env
var is just set so plugins can differentiate correctly. I don't unset
the variable since it shouldn't have any effect on `lightningd`
itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
pylightning: Added a tiny library for python plugins
It's flask inspired with the Plugin instance and decorators to add
methods to the plugin description.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
pylightning: Add option handling to the plugin library
Signed-off-by: Christian Decker <decker.christian@gmail.com>
pylightning: Migrate the test plugin to use the new wrapper
Should be a lot easier to see what happens :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
pylightning: Split log messages on newlines
This is just cosmetic, and makes things like tracebacks much easier to
read.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
pylightning: Merge option_values into options
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>

@cdecker cdecker force-pushed the plugin-6 branch from dcbb860 to 5dbc836 Dec 15, 2018

@cdecker

This comment has been minimized.

Copy link
Member

cdecker commented Dec 15, 2018

Interpreting the accept-review as an ACK, rebased and squashed.

ACK 5dbc836

@cdecker cdecker merged commit 42d8e07 into master Dec 15, 2018

3 checks passed

ackbot PR ack'd by cdecker
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment