ZeroMQ publisher plugin

[jarret]

A plugin that subscribes to notification and passes them through to be published to ZeroMQ endpoints. It utilizes Twisted and txZMQ to facilitate the publication. An additional script is provided to serve as an example for how to subscribe to the events.

The presentation was chosen to follow how bitcoind sets up its ZMQ publishing functionality.

NOTE: The plugin source file is given the name cl-zmq.py as opposed to zmq.py because the package dependencies import the official ZeroMQ python packages via from zmq import ... which is a namespace conflict with a source file named zmq.py.

[cdecker]

Very nice plugin, there is just the use of exec() which raises a couple of red flags to me. See comment for how you could work around that using native tools :-)

[cdecker]

This seems rather dangerous. I'd suggest creating a generic handler, use functools.partial to generate variants with the notification type bound and then using plugin.add_subscription to register them.

Something like the following would do I think:

def on_notification(notification_type, plugin, *args, **kwargs):
    if len(args) != 0:
        plugin.log("got unexpected args: {}".format(args), level="warn")
    reactor.callFromThread(publisher.publish_notification, "{}",
                           *args, **kwargs)

for nt in NOTIFICATION_TYPES:
    f = functools.partial(on_notification, nt)
    plugin.add_subscription(nt, f)

This would not call the python interpreter with a variable payload. In this case it's likely ok with your mechanism, since it just fills in the notification types, but in other cases this could be a security vulnerability since you execute code that was provided from outside.

[jarret]

Great suggestion. Defining code in a string also is kind of ugly and functools.partial() appears to be suited.

One issue I bumped into in the revised implementation is that Plugin._coerce_arguments() expects the callable object to have the __annotations__ attribute set, for which the partial object does not. To fix, it had to be manually set to an empty dict:

on.__annotations__ = {} # needed to please Plugin._coerce_arguments()

[cdecker]

Good catch, I'll add a check that should make that no longer necessary.

[cdecker]

I have patched pyln-client to check for the existence of __annotations__ before accessing it here ElementsProject/lightning@dfa7f48. Sadly it didn't make the v0.8.0 release window, so it'll be available from 0.8.1 onwards :-)

[cdecker]

It might be interesting to write a couple of tests by the way so that we don't accidentally break compatibility while working on c-lightning 😉