/lightning

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pylightning: Add a plugin framework #2161

Merged
merged 6 commits into from Dec 15, 2018

Conversation

Reviewers

@ZmnSCPxj ZmnSCPxj

@rustyrussell rustyrussell

@renepickhardt renepickhardt

Assignees

@cdecker cdecker

Labels
plugin
Projects
None yet
Milestone
No milestone
3 participants
@cdecker @rustyrussell @ZmnSCPxj
@cdecker
Copy link
Member

cdecker commented Dec 10, 2018 •
edited

This framework is very much inspired by flask, in that I wanted to make it as trivial as possible to get started. I had to jump through some hoops but I'm rather happy with the end result. A plugin with an option and a RPC passthrough is as simple as this:

from lightning import Plugin, monkey_patch
plugin = Plugin(autopatch=True)

@plugin.method("hello")
def hello(name, plugin):
    """Say hello"""
    greeting = plugin.get_option('greeting')
    plugin.log('{} {}'.format(greeting, name))
    return s

plugin.add_option('greeting', 'Hello', 'The greeting I should use.')
plugin.run()

It has support for monkey-patching stdout and stderr by default so that if run by lightningd it'll automatically wrap anything written to those streams into log notifications that get displayed nicely.

Depends-On: #2154

@cdecker cdecker added the plugin label Dec 10, 2018

@cdecker cdecker self-assigned this Dec 10, 2018

@cdecker cdecker requested review from rustyrussell and renepickhardt Dec 10, 2018

@cdecker cdecker force-pushed the plugin-6 branch from b694e5e to 792964c Dec 10, 2018

@ZmnSCPxj

ZmnSCPxj reviewed Dec 11, 2018
View changes

contrib/pylightning/lightning/plugin.py
try:
result = {
"jsonrpc": "2.0",
"result": self._dispatch(request),

This comment has been minimized.

Sign in to view
Copy link
@ZmnSCPxj

ZmnSCPxj Dec 11, 2018

Collaborator

It looks to me that this will result in "linear" dispatching, i.e. commands are handled one-at-a-time.

Would it be possible to have this handle dispatching in parallel, i.e. if a command stalls processing, another command can be processed?

Or is some Python limitation exists to prevent this, or it becomes complicated?

(maybe there can be a separate decoration for methods that might block)

This comment has been minimized.

Sign in to view
Copy link
@cdecker

cdecker Dec 11, 2018

Member

Good point, hadn't thought about it yet. I wanted to keep away from multi-threading as much as possible, but I'll try to see what I can come up with.

This comment has been minimized.

Sign in to view
Copy link
@cdecker

cdecker Dec 11, 2018

Member

I opened issue #2167 to track progress on this. Would love to hear feedback for the initial idea, which is future-inspired, but since the python futures are tightly bound to multi-threading I'm thinking of implementing my own AsyncResult instead.

This comment has been minimized.

Sign in to view
Copy link
@ZmnSCPxj

ZmnSCPxj Dec 11, 2018

Collaborator

This seems sensible.

@ZmnSCPxj ZmnSCPxj referenced this pull request Dec 11, 2018

Open

Implement an asynchronous dispatch to the plugin framework in pylightning #2167

@cdecker cdecker force-pushed the plugin-6 branch from 31eb298 to 17c69db Dec 13, 2018

@rustyrussell

rustyrussell approved these changes Dec 14, 2018
View changes

lightningd/plugin.c
@@ -895,6 +895,7 @@ void plugins_init(struct plugins *plugins, const char *dev_plugin_debug)
plugins->pending_manifests = 0;
uintmap_init(&plugins->pending_requests);

setenv("LIGHTNINGD_PLUGIN", "1", 1);

This comment has been minimized.

Sign in to view
Copy link
@rustyrussell

rustyrussell Dec 14, 2018

Contributor

Convenient! We could have also added a commandline option, but this is easier for plugins to grab in most languages.
contrib/pylightning/lightning/plugin.py
and stdin filedescriptor, so if we use them in some other way
(printing, logging, ...) we're breaking our communication
channel. This function
"""

This comment has been minimized.

Sign in to view
Copy link
@rustyrussell

rustyrussell Dec 14, 2018

Contributor

Did the rest of this comment get redirected to JSON? :)

This comment has been minimized.

Sign in to view
Copy link
@ZmnSCPxj

ZmnSCPxj Dec 14, 2018

Collaborator

It is a common problem that occurs when monkey patching system internals. The comment

This comment has been minimized.

Sign in to view
Copy link
@cdecker

cdecker Dec 14, 2018

Member

xD
contrib/pylightning/lightning/plugin.py Outdated
@@ -17,6 +17,7 @@ class Plugin(object):
def __init__(self, stdout=None, stdin=None, autopatch=True):
self.methods = {}
self.options = {}
self.option_values = {}

This comment has been minimized.

Sign in to view
Copy link
@rustyrussell

rustyrussell Dec 14, 2018

Contributor

I don't understand the distinction between options and option_values?

This comment has been minimized.

Sign in to view
Copy link
@cdecker

cdecker Dec 14, 2018

Member

options are the names of options that we registered with all their metadata, whereas option_values are the ones that were set by the init call from the plugin. We can merge them if desired of course.
@cdecker

This comment has been minimized.

Sign in to view
Copy link
Member

cdecker commented Dec 14, 2018

Merged options and option_values as suggested by @rustyrussell and fixed the comm

cdecker added some commits Dec 10, 2018

@cdecker
plugin: Set LIGHTNINGD_PLUGIN env var inform plugins 
It might be useful to take special precautions inside a plugin when
being run as a plugin (and not as a standalone executable). This env
var is just set so plugins can differentiate correctly. I don't unset
the variable since it shouldn't have any effect on `lightningd`
itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Verified
This commit was signed with a verified signature.
@cdecker cdecker Christian Decker
GPG key ID: 1416D83DC4F0E86D Learn about signing commits
324c1cc
@cdecker
pylightning: Added a tiny library for python plugins 
It's flask inspired with the Plugin instance and decorators to add
methods to the plugin description.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Verified
This commit was signed with a verified signature.
@cdecker cdecker Christian Decker
GPG key ID: 1416D83DC4F0E86D Learn about signing commits
1630528
@cdecker
pylightning: Add option handling to the plugin library 
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Verified
This commit was signed with a verified signature.
@cdecker cdecker Christian Decker
GPG key ID: 1416D83DC4F0E86D Learn about signing commits
2b7def9
@cdecker
pylightning: Migrate the test plugin to use the new wrapper 
Should be a lot easier to see what happens :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Verified
This commit was signed with a verified signature.
@cdecker cdecker Christian Decker
GPG key ID: 1416D83DC4F0E86D Learn about signing commits
34e8010
@cdecker
pylightning: Split log messages on newlines 
This is just cosmetic, and makes things like tracebacks much easier to
read.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Verified
This commit was signed with a verified signature.
@cdecker cdecker Christian Decker
GPG key ID: 1416D83DC4F0E86D Learn about signing commits
69d3d60
@cdecker
pylightning: Merge option_values into options 
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Verified
This commit was signed with a verified signature.
@cdecker cdecker Christian Decker
GPG key ID: 1416D83DC4F0E86D Learn about signing commits
Loading status checks…
5dbc836

@cdecker cdecker force-pushed the plugin-6 branch from dcbb860 to 5dbc836 Dec 15, 2018

@cdecker

This comment has been minimized.

Sign in to view
Copy link
Member

cdecker commented Dec 15, 2018

Interpreting the accept-review as an ACK, rebased and squashed.

ACK 5dbc836

@cdecker cdecker merged commit 42d8e07 into master Dec 15, 2018
3 checks passed

3 checks passed

ackbot PR ack'd by cdecker
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment