Rescan in pruned mode

[chris-belcher]

Electrum Personal Server works with Core's pruned mode, but that disables rescans. That means people who have a pruned full node and a wallet with historical transactions cannot easily adopt EPS.

I actually managed it in 2016 on JoinMarket which works the same way as EPS: JoinMarket-Org/joinmarket#431 (comment)

It is totally possible to have Core download blocks needed for rescanning while still keeping pruned enabled. One of these days I'll try to put some time into figuring out exactly how and writing a guide on it.

[MrPaz]

Just popping in to say this feature would be great. I use EPS on both a full node at home and pruned node on a work laptop, and often it's been a slight pain point not being able to use particular wallets on the work laptop.

[chris-belcher]

@MrPaz if you have an unpruned node you could rescan the wallet on it, and then copy over the wallet.dat file to your pruned node.

[MrPaz]

Oh really? So if I rescan the electrum wallet file on full node computer then put it on the pruned node computer it will have full history? That's nice.

What actually happens during rescan? ie What data is saved in the wallet file? Blocks with relevant transactions?

[chris-belcher]

Oh really? So if I rescan the electrum wallet file on full node computer then put it on the pruned node computer it will have full history? That's nice.

Yep it should do.

What actually happens during rescan?

Transactions and metadata (e.g. block hash which contains that tx) are saved in the wallet.dat file.

[NicolasDorier]

@chris-belcher you should fallback to scantxoutset if pruning is enabled.

[chris-belcher]

There's discussion in #27. Two issues of note are how are gap limits meant to work and how to stop/discourage users accidentally reusing addresses.

[chris-belcher]

In some situations the issue is bandwidth. People might have a pruned node snapshot (or from something like AssumeUTXO one day if that's added to Core). They have a problem obtaining the blocks to rescan because it costs a lot of bandwidth for them, but they don't mind a privacy tradeoff.

So a useful thing could be to have a feature in EPS which rescans by querying a blockchain explorer or public Electrum server, and then imports into Core's wallet using the RPC call importprunedfunds. Obviously this would dox all the user's addresses to the server, but there would be clear warnings. This would allow the user in that situation to use a wallet backed by their own full node.

The esplora API has merkleproofs ready to use https://github.com/Blockstream/esplora/blob/master/API.md#get-txtxidmerkleblock-proof The blockstream explorer at blockstream.info uses esplora, as does mempool.space

Note: the specter wallet developers publishes a pruned node snapshot at https://prunednode.today/, and btcpay has one at https://github.com/btcpayserver/btcpayserver-docker/tree/master/contrib/FastSync. of course it comes with a price though - you trust that whoever distributes the snapshot published an authentic copy of it, and could theoretically be fooled into accepting fake coins.

[Kixunil]

Would block filters help? We were discussing same problem at length and a solution is apparently being worked on.

[chris-belcher]

The total size of all the block filters is a few GB as well. Seems like it would make the bandwidth problem worse.

I heard a plan to use block filters in Core to make it faster to rescan stuff though, so maybe one day something like that could be useful.

[Kixunil]

I believe storing block filters is a reasonable tradeoff. That's also what the above mentioned solution proposes.