multi: add migrate-db command

[guggero]

This is work in progress and NOT recommended to be used with production/mainnet data!

[kiwiidb]

Using this code I was getting an error:
Runtime error: db connection set not initialized
I started looking around and found that you were probably missing this: https://github.com/lightningnetwork/lnd/blob/master/lncfg/db.go#L140

So I added a quick patch: getAlby@0ffcae3
(should be a config param probably).

And then I was able to do the migration. The last logs were:

2022-07-14 10:41:42.534 LNDINIT: Opened destination DB
2022-07-14 10:41:42.535 LNDINIT: Checking tombstone marker on source DB
2022-07-14 10:41:42.536 LNDINIT: Checking DB version of source DB
2022-07-14 10:41:42.537 LNDINIT: Checking if migration was already applied to target DB
2022-07-14 10:41:42.540 LNDINIT: Starting the migration to the target backend
2022-07-14 10:41:42.540 LNDINIT: Copying top-level bucket 'waddrmgr'
2022-07-14 10:41:48.981 LNDINIT: Committing bucket 'waddrmgr'
2022-07-14 10:41:48.983 LNDINIT: Copying top-level bucket 'wtxmgr'
2022-07-14 10:41:49.016 LNDINIT: Committing bucket 'wtxmgr'
2022-07-14 10:41:49.017 LNDINIT: Creating 'wallet created' marker
2022-07-14 10:41:49.024 LNDINIT: Committing 'wallet created' marker

It seems everything has migrated succesfully, so I would say that a log line indicating this would also be useful.

[twofaktor]

Hello, what happened with this PR? Have been it abandoned? I think this is very useful to migrate an existing bbolt to postgres and following this guide

Thanks!

[ziggie1984]

This is an important PR we need to focus on to allow nodes to finally migrate from kv to native sql. cc @saubyk

[Roasbeef]

sqlite should be added here: https://github.com/lightninglabs/lndinit/pull/21/files#diff-00eb92ba2060dddcdbce2dba1dc551065557b5ebcf98e47e70946e1b50ffd243R453

The top-level bucket structure might also have changed somewhat, so we should update that aspect.

One other thing is we'll need to figure out a way to ensure that all the data has truly been migrated (eg: we don't forget some other top level bucket recently added).

[saubyk]

Still testing, but wanted to note some comments

[saubyk]

Need support for networks like testnet and regtest as well

[saubyk]

shouldn't the link be: https://github.com/lightningnetwork/lnd/blob/master/docs/postgres.md?

[saubyk]

Should be a WARNING/CAUTION instead of NOTE

[saubyk]

Should end with a section advising user to run the migrated node with the updated db config settings

[ziggie1984]

One other thing is we'll need to figure out a way to ensure that all the data has truly been migrated (eg: we don't forget some other top level bucket recently added).

so you mean having the db files constant here (channeldb, towerdb etc) is not enough but we need to also make sure every new bucket is in the new db ? Hmm not sure I understand this one? Because as long as we iterate through each bucket in the db file, there shouldn't be a problem that we miss a bucket ?