New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RPC plugin management #2771
RPC plugin management #2771
Conversation
4ca10ce
to
bf41958
Compare
Can we have a top-level "plugin" command, with "stop", "start", "adddir"? Then we can stop and start them individually or all? And list of course... |
Design-wise, I would suggest adding a new function plugin_reload_all
that correctly performs the reload of plugins, which would be async, and create a new plugin_control.c
file that contains the RPC commands.
You also need to consider what happens to ongoing plugin-based commands. Would not it be better if we enter a "shutting down" mode where existing plugin-provided commands are allowed to complete, new plugin-provided commands are delayed until after reload, and once current plugin commands have completed, only then kill and reload plugins?
Consider that we may eventually move existing commands like withdraw
to a plugin (using txprepare
and txbroadcast
). There is always a race condition where client is executing a withdraw and then another client executes a reload_plugin
, it would be a regression in behavior if before withdraw
is a plugin this is not a problem but if withdraw
is moved to a plugin then it becomes problematic if we reload_plugin
while a withdraw
command is ongoing.
Even fundchannel
can eventually be moved to a plugin (it can eventually be implemented with fundchannel_start
, txprepare
, fundchannel_complete
, and txbroadcast
).
Yes, I'm not sure but I think I can do that. @ZmnSCPxj thank you for the review
I agree for the seperated file. Do you recommend this command in addition to the one Rusty proposed ?
I may add a check, even if it seems weird to me that someone would |
According to the above comments by @rustyrussell and @ZmnSCPxj :
|
136a98d
to
c602cde
Compare
Rebased to move Since the load from the default plugin directory is now done each time we call a |
e8402f5
to
726d6b1
Compare
The travis failure is due to Backtrace to avoid pull and compilation
The backtrace after having quiet the warning notifications ( 2019-06-27T20:52:39.201Z lightningd(23821): --------------------------------------------------
2019-06-27T20:52:39.201Z lightningd(23821): Server started with public key 023bafa606486c3066f71d98f385dcfb24742191589922bcbea49373c21c4e7457, alias PEEVEDSOURCE-623-g726d6b1-modded (color #023baf) and lightningd v0.7.0-623-g726d6b1-modded
Already in transaction from lightningd/jsonrpc.c:653
2019-06-27T20:52:40.251Z lightningd(23821): Already in transaction from lightningd/jsonrpc.c:653
lightningd: FATAL SIGNAL 6 (version v0.7.0-623-g726d6b1-modded)
0x55af55bb0aa8 send_backtrace
common/daemon.c:40
0x55af55bb0b4e crashdump
common/daemon.c:53
0x7fa88ffdc83f ???
???:0
0x7fa88ffdc7bb ???
???:0
0x7fa88ffc7534 ???
???:0
0x55af55b8be73 fatal
lightningd/log.c:622
0x55af55bdd0ae db_begin_transaction_
wallet/db.c:637
0x55af55ba990e sd_msg_read
lightningd/subd.c:409
0x55af55bf64c9 next_plan
ccan/ccan/io/io.c:59
0x55af55bf7046 do_plan
ccan/ccan/io/io.c:407
0x55af55bf7084 io_ready
ccan/ccan/io/io.c:417
0x55af55bf90d9 io_loop
ccan/ccan/io/poll.c:445
0x55af55ba778d plugins_init
lightningd/plugin.c:969
0x55af55ba8042 json_plugin_control
lightningd/plugin_control.c:65
0x55af55b8729b parse_request
lightningd/jsonrpc.c:654
0x55af55b876d5 read_json
lightningd/jsonrpc.c:752
0x55af55bf64c9 next_plan
ccan/ccan/io/io.c:59
0x55af55bf7046 do_plan
ccan/ccan/io/io.c:407
0x55af55bf7084 io_ready
ccan/ccan/io/io.c:417
0x55af55bf90d9 io_loop
ccan/ccan/io/poll.c:445
0x55af55b83b0d io_loop_with_timers
lightningd/io_loop_with_timers.c:24
0x55af55b89d43 main
lightningd/lightningd.c:827
0x7fa88ffc909a ???
???:0
0x55af55b720d9 ???
???:0
0xffffffffffffffff ???
???:0
2019-06-27T20:52:40.258Z lightningd(23821): FATAL SIGNAL 6 (version v0.7.0-623-g726d6b1-modded)
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: common/daemon.c:45 (send_backtrace) 0x55af55bb0afe
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: common/daemon.c:53 (crashdump) 0x55af55bb0b4e
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: (null):0 ((null)) 0x7fa88ffdc83f
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: (null):0 ((null)) 0x7fa88ffdc7bb
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: (null):0 ((null)) 0x7fa88ffc7534
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/log.c:622 (fatal) 0x55af55b8be73
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: wallet/db.c:637 (db_begin_transaction_) 0x55af55bdd0ae
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/subd.c:409 (sd_msg_read) 0x55af55ba990e
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55af55bf64c9
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55af55bf7046
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55af55bf7084
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/poll.c:445 (io_loop) 0x55af55bf90d9
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/plugin.c:969 (plugins_init) 0x55af55ba778d
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/plugin_control.c:65 (json_plugin_control) 0x55af55ba8042
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/jsonrpc.c:654 (parse_request) 0x55af55b8729b
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/jsonrpc.c:752 (read_json) 0x55af55b876d5
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55af55bf64c9
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55af55bf7046
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55af55bf7084
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: ccan/ccan/io/poll.c:445 (io_loop) 0x55af55bf90d9
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55af55b83b0d
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: lightningd/lightningd.c:827 (main) 0x55af55b89d43
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: (null):0 ((null)) 0x7fa88ffc909a
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: (null):0 ((null)) 0x55af55b720d9
2019-06-27T20:52:40.258Z lightningd(23821): backtrace: (null):0 ((null)) 0xffffffffffffffff
Log dumped in crash.log.20190627205240
Aborted |
The root reason is that every command is already inside a sqlite3 db transaction. We do everything inside db transactions, only exiting when returning to the mainloop (e.g. async operations). What's happening is that you are trying to run a mainloop within a mainloop, so we try to make a transaction inside a transaction, which isn't supported (at least by our own code; sqlite3 might work, our code might not). Like I suggested, you should split In particular, you should not do an So For For |
a9a834a
to
3ec8de2
Compare
@ZmnSCPxj thank you the support : I had to modify I added the |
9135b53
to
bd14d0b
Compare
@ZmnSCPxj do the changes seem good to you ? |
Would rpc.plugin.start() etc be more pythony? ie. make plugin like a command namespace... somehow? |
I think this was about dynamic json fields unpacking to function arguments which breaks plugins made with Pylightning if we add a new field in JSONRPC params (in |
Rebased on top of master and resolve CHANGELOG conflicts. Added new brag lines for |
@@ -12,6 +12,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | |||
|
|||
- build: now requires `python3-mako` to be installed, i.e. `sudo apt-get install python3-mako` | |||
- plugins: a new notification type `invoice_payment` (sent when an invoice is paid) has been added | |||
- rpc: a new rpc command is added, `plugin`. It allows one to manage plugins without restarting `lightningd`. | |||
- plugins: a new boolean field is added to the `init`'s `configuration`, `startup`. It allows a plugin to know if it has been started on `lightningd` startup. | |||
- plugins: a new boolean field can be added to a plugin manifest, `dynamic`. It allows a plugin to tell if it can be started or stopped "on-the-fly". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll probably collapse these three lines into "plugins: dynamic plugin support", which is the better summary for this level IMHO.
(I do a CHANGELOG.md cleanup sweep as part of the release process, so I won't hold up the merge for this!)
…ugin_kill' non static
…lit 'plugins_init' This adds a 'configured' boolean member to the plugin struct so that we can add plugins to ld->plugins' list and differenciate fresh plugins. This also adds 'plugins_start' so that new plugins can be started without calling 'plugins_init' and running an io loop
This adds 'plugin_unregister_hook' and 'plugin_unregister_hook_all' functions to unregister a given hook a plugin registered, or all hooks a plugin registered for. Since hooks can only be registered once, it's useful in the case a new plugin is added which would be prefered for hook registration over an already loaded plugin.
This adds a new pair of files : lightningd/plugin_control, along with a new RPC command : 'plugin'. This command can be used to manage plugins without restarting lightningd: lightning-cli plugin start helloworld.py lightning-cli plugin stop helloworld.py
…p' field to init This lets a plugin specify whether it can be restarted, and to know if it is started at lightningd startup
'init' and 'getmanifest'
Trivial rebase to resolve CHANGELOG.md conflict. Ack a298c50 |
Can someone add the |
This PR adds a new RPC command :reloadplugins
, which will initialize and configure each plugin from the default directory (~/.lightning/plugins/
) -or from another passed as parameter- that were not already loaded at lightningd startup.This PR has mutated to now add a
plugin
RPC command : #2771 (comment).This permits to manage plugins without having to restart lightningd.