Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
multi: implement new safe static channel backup and recovery scheme, RPCs, and cli commands #2313
In this PR, we implement a new safe scheme for static channel backups (SCB's) for
Once this PR is merged, given their seed and the latest back up file, the user will be able to recover both their on-chain funds, and also funds that are fully settled within their channels. By "fully settled" we mean funds that are in the base commitment outputs, and not HTLCs. We can only restore these funds as right after the channel is created, we have all the data required to make a backup. In contrast, in order to resolve HTLCs, we would also need to update the backup state with each new channel update, which is tricky to do without additional infrastructure. This infrastructure will be built out in the near future, but until then we have this scheme which will also be a fall back in the scenario that any higher level mechanisms fail.
At a later point, we also plan to propose this backup scheme as an addition to the spec, as even with the change to make the "to self" outputs static, we still need this SCB information in order to restore user funds. Additionally, the current serialization format is a bit up in the air. Atm, we use the same "codec" as we do within the wire protocol for the BOLT specs. However, we'll likely move to a TLV (type-length-value) format as it's extremely flexible and allows us to add/remove fields in the future once we gain new channel types, or modifications are made in the protocol that warrant a change to the backup format. Most importantly, if
Skipping the backup flow for a second, given their 24-word
Backup + Recovery Methods
This PR exposes multiple safe ways to backup and recover a channel. We expect only one of them to be used primarily by unsophisticated end users, but have provided other mechanisms for more advanced users and business that already script
First, the easiest method for backup+recovery. After this PR,
The second mechanism is via the new
Finally, users are able to request a backup of a single channel, or all the channels via the cli and RPC methods. Here's an example, of a few ways users can obtain backups, see the PR for full details:
Static Channel Backup Scheme
For encryption, we utilize
For key generation, in order to ensure the user only needs their passphrase and the backup file, we utilize the existing keychain to derive a private key. In order to ensure that at we don't force any hardware signer to be aware of our crypto operations, we instead opt to utilize a public key that will be hashed to derive our private key. The assumption here is that this key will only be exposed to this software, and never derived as a public facing address.
"First, the easiest method for backup+recovery. After this PR, lnd will maintain a channels.backup file in the same location that we store all the other files. .."
Can a dedicated folder be used? If it is mounted with sshfs or nfs, the channels.backup and channel.db files can be separated into different machine.
Alrighty, I've broken this PR up into 5 distinct PR's. Each new PR depends on the prior PR. As a result, they can go in one by one and be reviewed in smaller units, rather than waiting for the final dependents of this larger PR to be finalized. I'll keep this one as is though as it has the full description, and also builds allowing users to experiment with the set of commands. Once the final PR is ready for review (as all the prior PRs have been merged), I'll rebase this on on top of that, so everyone can use this as a central point of end to end testing.
…recovered chan In this commit, we modify the `closeObserver` to fast path the DLP dispatch case if we detect that the channel has been restored. We do this as otherwise, we may inadvertently enter one of the other cases erroneously, causing us to now properly look up their dlp commitment point.
In this commit, we convert the server's Start/Stop methods to use the sync.Once. We do this in order to fix concurrency issues that would allow certain queries to be sent to the server before it has actually fully start up. Before this commit, we would set started to 1 at the very top of the method, allowing certain queries to pass before the rest of the daemon was had started up. In order to fix this issue, we've converted the server to using a sync.Once, and two new atomic variables for clients to query to see if the server has fully started up, or is in the process of stopping.
During the restore process, it may be possible that we have already heard about our prior edge from a node on the network (or our channel peers). As a result, we shouldn't exit if this happens, and instead should continue with the rest of the restoration process.
In this commit, we modify the `RestoreNodeWithSeed` and `RestartNode` methods to also accept an SCB. This will be useful in new integration tests to properly exercise the various restore/restart scenarios using static channel backups.
…t to new func In this commit, we modify the core testDataLossProtection test to extract the primary DLP assertion logic into a new function. We do this, as the upcoming SCB tests will fallback to this test after some initial set up.
In this commit, we add 4 new itests for exercising the SCB restore process via 4 primary scenarios: recover from backup using RPC, recover from file using RPC, recover channels during init/creation, recover channels during unlock. With all fields populated there're a total of 24 new scenarios to cover. At the time of authoring of this commit, the other scenarios (bits are: initiator, updates, private) have been left out for now, as they increased the run time of the integration tests significantly.
Any chance you can include what the commands are to restore (exact syntax), and what the expected outputs would be (just and example)? Considering how important this is, just guess and 'tying to figure it out' may not be the best idea.
From my understanding, this isn't actually a "back up" of the channels, and is instead a "channel funds recovery mechanism". Correct? If you restored using this, you'd have a node with zero channels, and would have to start open channels from scratch. Correct?
Check out the PR description, more docs will be provided later.…
On Mon, Apr 1, 2019, 5:23 PM ZapUser77 ***@***.***> wrote: Any chance you can include what the commands are to restore (exact syntax), and what the expected outputs would be (just and example)? Considering how important this is, just guess and 'tying to figure it out' may not be the best idea. From my understanding, this isn't actually a "back up" of the channels, and is instead a "channel funds recovery mechanism". Correct? If you restored using this, you'd have a node with zero channels, and would have to start open channels from scratch. Correct? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#2313 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA87Lk9FdliyMFr1ImFE7wRLrAHHFsSZks5vcqL7gaJpZM4ZMeKM> .
Would it be possible to create a "backup" manually, in the scenario where a node is lost and accessing the original channels.backup file isn't accessible anymore?
Put in another way, given that you know the