validation: assumeutxo params mainnet

[Sjors]

This adds snapshot parameters for mainnet block 840,000.

Depends on #29612

You can generate the snapshot yourself using ./contrib/devtools/utxo_snapshot.sh or download my torrent:

• torrent: magnet:?xt=urn:btih:55cb2ea9b6c48aa11d676de871f9639e3fa3dd7e&dn=utxo-840000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969

It would be a good idea to test:

1. That you can produce the same snapshot file, sha256 sum:

9db649785fcddd8b1fe83a5009fe5c9be1901e8c322298cacc8a2d96ace4807e utxo-840000.dat

2. That the snapshot works

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Stale ACK	pablomartin4btc

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

[Sjors]

Updated hash_serialized after #28685.

I only tested that the loadtxoutset command finishes.

@pablomartin4btc I found a different hash than you... Sjors@24deb20#commitcomment-130847775

[pablomartin4btc]

tACK 0800be2

./contrib/devtools/utxo_snapshot.sh 800000 - ./src/bitcoin-cli -datadir=${AU_DATADIR}
Rewinding chain back to height 800000 (by invalidating 00000000000000000000e26b239cf19ec7ace5edd9694d51a3f6933247720947); this may take a while
Generating txoutset info...
{
  "height": 800000,
  "bestblock": "00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054",
  "txouts": 111535121,
  "bogosize": 8443325590,
  "hash_serialized_3": "6e63799cac0ab45f94789f97692caf691ee66625e920cbf2525fceb24509adda",
  "total_amount": 19437286.83643360,
  "transactions": 72936026,
  "disk_size": 7264997650
}
Restoring chain to original height; this may take a while

I incorrectly took the muhash value from gettxoutsetinfo (without running the utxo_snapshot.sh which also performs invalidateblock right before).

[Sjors]

@pablomartin4btc thanks for checking that you get the same snapshot. In addition it would be good to test if you're able to load the snapshot and complete the background sync.

[pablomartin4btc]

In addition it would be good to test if you're able to load the snapshot and complete the background sync.

I've already checked IBD completed fine with[background validation]finished, but with the previoushash_serialized, as I wanted to verify that #28698 was only happening before the IBD completion.

I'll run all the process again with the newhash_serializedvalue.

[pablomartin4btc]

Tested loadtxoutset a few times (pending IBD to finish) with coinstatsindex & blockfilterindex, and without them. I've also validated that the sha256sum corresponds to the one obtained from the downloaded file.

Now, having a pruned node with IBD completed, from which I got the m_assumeutxo_data mentioned in my previous review, when I run ./contrib/devtools/utxo_snapshot.sh with a filename in order to perform the actual dumptxoutsetwithin the script, I got a node crash with ERROR: ReadBlockFromDisk: OpenBlockFile failed for FlatFilePos(nFile=-1, nPos=0), which instance I had to kill since it was unresponsive. I'm not sure if this situation was due to the invalidateblock performed twice by runnning the utxo_snapshot script (one to get the info from gettxoutsetinfo and the second for the actual dump), the pruned node or a combination of all three. Starting the node again without blockfilterindex (option given by the node before doing the suggested -reindex or -reindex-chainstate which doesn't work on pruned nodes), made it worse till getting !setBlockIndexCandidates.empty(). I see there are a few open issues regarding invalidateblock & setBlockIndexCandidates so before investigating further I'd like to know first if the dumptxoutset is expected to work from a fully synced pruned node?

[Sjors]

As discussed offline, the utxo_snapshot.sh script should have a sanity check to make the node isn't pruned below the snapshot height. On a pruned node you can only make a snapshot for a block that hasn't been pruned yet.

[pablomartin4btc]

As discussed offline, the utxo_snapshot.sh script should have a sanity check to make the node isn't pruned below the snapshot height. On a pruned node you can only make a snapshot for a block that hasn't been pruned yet.

I agree with it, mentioned on a #27845 that's touching the script ATM.

In the testing I performed and mentioned above the pruned node was above the snapshot height (800'000):

  "pruned": true,
  "pruneheight": 600510,
  "automatic_pruning": true,
  "prune_target_size": 20971520000,

[Sjors]

I plan to make a fresh snapshot after #26045 lands.

[fjahr]

Didn't see this one until the conflict popped up. I think we can move to 830,000 for v27.0 and I am unsure if we need to wait for #26045. I will bring it up in the next meeting.

[Sjors]

I switched to the halving block! Also dropped all prerequisites but #26045 from the description, but see conceptual discussion in #29616.