Prune Node Rescan Project Tracking

[furszy]

This issue will be edited frequently to reflect the current status of the project.

What should I review now?


👇 👇 👇 👇
-- #30118 --
☝️ ☝️ ☝️ ☝️

The goal of this project is to enable complete blockchain rescan capability on prune nodes, allowing these storage-limited nodes to import new wallet descriptors and update existing ones at any time, as well as loading external and un-synced wallets with a birth-time older than 48 hours. This, in turn, enables the monitoring of new addresses/scripts and tracking balance changes without the need to repeatedly download the entire blockchain.

The first project milestone introduces the feature for prune nodes that construct and maintain a local block filters set database. The node will request and keep track of the missing historical blocks that match the wallet scripts filters set to subsequently perform the rescan.

The following sections are ordered by their PRs dependencies:

Net: Single Block Requests Tracking System and On Download Failure Retry Mechanism

• p2p: adaptive connections services flags #28170
• p2p: make block download logic aware of limited peers threshold #28120
• test: improve robustness of connect_nodes() #30118
• net: introduce block tracker to retry to download blocks after failure #27837

Indexes Stability and Performance Improvements

• index: make startup more efficient #27607
• index: prevent race by calling 'CustomInit' prior setting 'synced' flag #27720
• index: block filters sync, reduce disk read operations by caching last header #28955
• index: blockfilter initial sync speedup, parallelize process #26966

Wallet Rescan (depends on the net section)

• wallet: improve IBD sync time by skipping block scanning prior birth time #27469
• Cache wallet filters' set and connect it to the current wallet rescan functionality. (Pending)
• Wallet: implement "rescan status" class to track the progress and retrieve it via an RPC command. (Pending)
• Wallet: introduce prune node rescan functionality. (Pending - depends on net changes)

Issues Solved By This Work

• Allow for rescan using block filters with pruned node #21267
• Make rescans faster #23727
• Use of a wallet shouldn't be blocked in prune mode ("wallet loading failed... beyond pruned data") #27188
• Allow getblockfrompeer to use any peer #27652
• Optimization: utilize 100% resources of a computer in long operations like rescan/initial block download #27939
• BIP157 / neutrino mode sparrowwallet/sparrow#1111 (comment)

[chrisguida]

Does scanblocks work on pruned nodes yet? If not, does this project aim to add that capability?

[furszy]

Does scanblocks work on pruned nodes yet? If not, does this project aim to add that capability?

scanblocks always worked on pruned nodes. Just need to enable -blockfilterindex since the node's inception.

[chrisguida]

Excellent, thanks!!

[chrisguida]

What's the process for building the blockfilterindex on an already-synced pruned node? Enable blockfilterindex, reindex?

[furszy]

What's the process for building the blockfilterindex on an already-synced pruned node? Enable blockfilterindex, reindex?

Yes. For further node support, please come to IRC or use bitcoin.stackexchange. This is a project tracking issue.

[shovas]

Here from #27188. I'm presently experimenting with wallets for bitcoin, litecoin, dogecoin, and bitcoin cash using linux cli tools so if I can help I'll try.