Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

script, assumeutxo: Enhance validations in utxo_snapshot.sh #28852

Merged

Conversation

pablomartin4btc
Copy link
Member

@pablomartin4btc pablomartin4btc commented Nov 11, 2023

This PR resolves #27841 and some more:

  • Ensure that the snapshot height is higher than the pruned block height when the node is pruned (Suggested by @Sjors here).

  • Validate the correctness of the file path and check if the file already exists (@hazeycode's #27845).

  • Make network activity disablement optional for the user (Suggested by @Sjors here and here).

  • Ensure the reconsiderblock command is triggered on exit (@hazeycode's same PR as above), even in the case of user interruption (Ctrl-C).

In order to perform some testing please follow the instructions in the description of previous @hazeycode's PR #27845.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 11, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK Sjors, ryanofsky
Concept ACK BrandonOdiwuor

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

@pablomartin4btc pablomartin4btc marked this pull request as draft November 11, 2023 16:36
@pablomartin4btc pablomartin4btc force-pushed the devtools-improve-utxo_snapshot branch 2 times, most recently from 156cd0c to 579fcae Compare November 11, 2023 16:55
@pablomartin4btc pablomartin4btc marked this pull request as ready for review November 11, 2023 17:00
@Sjors
Copy link
Member

Sjors commented Nov 13, 2023

Concept ACK

Copy link
Member

@Sjors Sjors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions after testing and reviewing 3cb390f.

contrib/devtools/utxo_snapshot.sh Outdated Show resolved Hide resolved
contrib/devtools/utxo_snapshot.sh Show resolved Hide resolved

function cleanup {
(>&2 echo "Restoring chain to original height; this may take a while")
${BITCOIN_CLI_CALL} reconsiderblock "${PIVOT_BLOCKHASH}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that once invalidateblock has been called, this won't stop it. It will go all the way back and then forward again.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as in the previous PR #27845. We could add more context in the output message and perhaps we could update the documentation as well:

The utility script
`./contrib/devtools/utxo_snapshot.sh` may be of use.

@Sjors
Copy link
Member

Sjors commented Nov 13, 2023

By the way, at some point when a bash script gets very complicated, it may be better to convert it to Python.

@pablomartin4btc
Copy link
Member Author

By the way, at some point when a bash script gets very complicated, it may be better to convert it to Python.

I do agree, but so far I think it's manageable. I'm happy to convert it if more reviewers see other issues there.

- Ensure that the snapshot height is higher than the pruned block height when the node is pruned.
- Validate the correctness of the file path and check if the file already exists.
- Make network activity disablement optional for the user.
- Ensure the reconsiderblock command is triggered on exit, even in the case of user interruption (Ctrl-C).

Co-authored-by: Chris Heyes <22148308+hazeycode@users.noreply.github.com>
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
@pablomartin4btc
Copy link
Member Author

Addressed @Sjors's feedback.

cc @jamesob as he reviewed #27845, @ryanofsky & @theStack

@Sjors
Copy link
Member

Sjors commented Nov 14, 2023

tACK 11b7269

Copy link
Contributor

@BrandonOdiwuor BrandonOdiwuor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Copy link
Contributor

@BrandonOdiwuor BrandonOdiwuor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested ACk 11b7269

  1. Successfully tested error handling added and chainstate can be automatically recovered if an error occurs e.g when the snapshot file already exists or user interruption by ctrl + c
  2. Successfully tested the option to enable or disable network activity

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 11b7269

# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
export LC_ALL=C

set -ueo pipefail

NETWORK_DISABLED=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "script: Enhance validations in utxo_snapshot.sh" (11b7269)

Assignment seems a little out of place here, would suggest moving it below PRUNE=... with other variable assignments.

@DrahtBot DrahtBot requested review from BrandonOdiwuor and removed request for BrandonOdiwuor December 4, 2023 22:03
@ryanofsky ryanofsky merged commit 1c8893b into bitcoin:master Dec 4, 2023
16 checks passed
@ryanofsky
Copy link
Contributor

Note: this actually had 3 acks. DrahtBot didn't count #28852 (review) due to capitalization

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stuck chainstate when utxo_snapshot.sh fails
5 participants