New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[rpc] getblockchaininfo: add size_on_disk, prune_target_size #11367
[rpc] getblockchaininfo: add size_on_disk, prune_target_size #11367
Conversation
If #11359 gets in it might be nice to have the prune hwm in there too, but I didn't want to add the dependency. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
concept ACK. One nit inline.
Consider adding an explicit test for the getblockchaininfo
RPC method to test/functional/blockchain.py
.
src/rpc/blockchain.cpp
Outdated
@@ -1137,6 +1137,8 @@ UniValue getblockchaininfo(const JSONRPCRequest& request) | |||
" \"chainwork\": \"xxxx\" (string) total amount of work in active chain, in hexadecimal\n" | |||
" \"pruned\": xx, (boolean) if the blocks are subject to pruning\n" | |||
" \"pruneheight\": xxxxxx, (numeric) lowest-height complete block stored\n" | |||
" \"disk_size\": xxxxxx, (numeric) the estimated size of the block and undo files on disk\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: perhaps disk_usage
or size_on_disk
?
Also consider placing this above pruned
so all of the pruning-related fields are grouped.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also align description.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO we should test the response result. In this case, at least, check that the key exists and value is a number. cc @jnewbery
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, @jnewbery already recommended a test! 👍
src/rpc/blockchain.cpp
Outdated
@@ -1180,6 +1182,7 @@ UniValue getblockchaininfo(const JSONRPCRequest& request) | |||
obj.push_back(Pair("verificationprogress", GuessVerificationProgress(Params().TxData(), chainActive.Tip()))); | |||
obj.push_back(Pair("chainwork", chainActive.Tip()->nChainWork.GetHex())); | |||
obj.push_back(Pair("pruned", fPruneMode)); | |||
obj.push_back(Pair("disk_size", CalculateCurrentUsage())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing lock of cs_LastBlockFile
.
Test for |
a2d851f
to
62094bf
Compare
002127c
to
96f5558
Compare
96f5558
to
ae3cbb9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one more documentation nit. Otherwise looks good to me
src/rpc/blockchain.cpp
Outdated
@@ -1135,8 +1135,10 @@ UniValue getblockchaininfo(const JSONRPCRequest& request) | |||
" \"mediantime\": xxxxxx, (numeric) median time for the current best block\n" | |||
" \"verificationprogress\": xxxx, (numeric) estimate of verification progress [0..1]\n" | |||
" \"chainwork\": \"xxxx\" (string) total amount of work in active chain, in hexadecimal\n" | |||
" \"size_on_disk\": xxxxxx, (numeric) the estimated size of the block and undo files on disk\n" | |||
" \"pruned\": xx, (boolean) if the blocks are subject to pruning\n" | |||
" \"pruneheight\": xxxxxx, (numeric) lowest-height complete block stored\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The help text for pruneheight
should also include "(only present if pruning is enabled)"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
4a61af9
ae3cbb9
to
4a61af9
Compare
src/validation.h
Outdated
@@ -156,6 +156,7 @@ struct BlockHasher | |||
|
|||
extern CScript COINBASE_FLAGS; | |||
extern CCriticalSection cs_main; | |||
extern CCriticalSection cs_LastBlockFile; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather not expose more variables that are private to validation to the outside world.
Perhaps you can instead create a wrapper around CalculateCurrentUsage
, which just grabs cs_LastBlockFile
and calls the internal one, and expose that? That way the RPC code wouldn't need to know about the existence of that lock eve.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just lock cs_LastBlockFile
in CalculateCurrentUsage
? It's a recursive mutex and IMO should be locked where it's needed (BTW the same for remaining locks...).
src/validation.h
Outdated
@@ -156,6 +156,7 @@ struct BlockHasher | |||
|
|||
extern CScript COINBASE_FLAGS; | |||
extern CCriticalSection cs_main; | |||
extern CCriticalSection cs_LastBlockFile; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not just lock cs_LastBlockFile
in CalculateCurrentUsage
? It's a recursive mutex and IMO should be locked where it's needed (BTW the same for remaining locks...).
src/rpc/blockchain.cpp
Outdated
obj.push_back(Pair("pruned", fPruneMode)); | ||
if (fPruneMode) { | ||
CBlockIndex *block = chainActive.Tip(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CBlockIndex* block = ...
;
src/rpc/blockchain.cpp
Outdated
@@ -1168,7 +1170,7 @@ UniValue getblockchaininfo(const JSONRPCRequest& request) | |||
+ HelpExampleRpc("getblockchaininfo", "") | |||
); | |||
|
|||
LOCK(cs_main); | |||
LOCK2(cs_main, cs_LastBlockFile); // cs_LastBlockFile for CalculateCurrentUsage() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As @sipa already commented, CalculateCurrentUsage()
should be responsible for concurrency locking (not the calling code part).
src/rpc/blockchain.cpp
Outdated
" \"pruned\": xx, (boolean) if the blocks are subject to pruning\n" | ||
" \"pruneheight\": xxxxxx, (numeric) lowest-height complete block stored\n" | ||
" \"pruneheight\": xxxxxx, (numeric) lowest-height complete block stored (only present if pruning is enabled)\n" | ||
" \"prune_target_size\": xxxxxx, (numeric) the target size used by pruning (only present if pruning is enabled)\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indentation seems wrong (others need to space-up).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I followed the example of 'verificationprogress'; can indent the others further if you think that makes sense.
Concept ACK |
4a61af9
to
071879a
Compare
Rebased on master, moved the lock into CalculateCurrentUsage, Travis failed on one platform (iirc linux64) on a timeout in p2p-segwit last time. Not sure why. We'll see how it goes. edit: looks good. |
utACK 071879a6263aa286e166f9cf07f409e6df35bc02 |
I've just tested this with
That's because of this section in
Returning the max int for |
Hmmm. Yes, that isn't desirable behaviour. Options I can see: b) fixes the RPC without having to touch pruning code but otherwise seems a bit messy to me. |
Or instead of b) just see if gArgs.GetArg("-prune", 0) is 1? |
I think just testing the value of |
071879a
to
3e827c4
Compare
|
Please rebase with #11370. |
…ic_pruning Fix pruneheight help text. Move fPruneMode block to match output ordering with help text. Add functional tests for new fields in getblockchaininfo. Github-Pull: bitcoin#11367 Rebased-From: b7dfc6c
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
This commit is extracted from upstream PR: bitcoin/bitcoin#11367
This commit is extracted from upstream PR: bitcoin/bitcoin#11367
This commit is extracted from upstream PR: bitcoin/bitcoin#11367
This commit is extracted from upstream PR: bitcoin/bitcoin#11367
This commit is extracted from upstream PR: bitcoin/bitcoin#11367
This commit is extracted from upstream PR: bitcoin/bitcoin#11367
* Backport size_on_disk to RPC call getblockchaininfo. This commit is extracted from upstream PR: bitcoin/bitcoin#11367 * Add size_on_disk test Co-authored-by: Simon <simon@bitcartel.com> * Rename methods to include Sprout * Add benchmark for decrypting sapling notes * Move reusable Sapling test setup to utiltest * Move test SaplingNote creation to utiltest * Add test method for generating master Sapling extended spending keys * Include Sapling transactions in increment note witness benchmark * Prevent header from being included multiple times * benchmarks do not require updating network parameters * FakeCoinsViewDB can inherit directly from CCoinsView * Add a method for generating a test CKey * Change to t->z transaction and create separate benchmark for sapling * Renaming and other minor cleanup * Improve some error messages when building a transaction fails * redirect and update source documentation * Update OpenSSL from 1.1.0h to 1.1.1a. #3786 * Update boost from v1.66.0 to v1.69.0. #3786 * Update Rust from v1.28.0 to v1.32.0. #3786 * Update Proton from 0.17.0 to 0.26.0. #3816, #3786 * Patch Proton for a minimal build. #3786 * Update of copyright year to 2019 * Add Sapling benchmarks to benchmark runner * Add missing author aliases * Correcting logo on README * Simplify DisconnectBlock arguments/return value DisconnectBlock currently has a complicated interface: Situation Return value pfClean != nullptr pfClean == nullptr All good: true true Failure: false false Unclean rewind: true false with *pfClean=false Change this to return a tristate enum instead. As an added bonus, remove the ValidationState& argument which was unused. * Add Sprout support to TransactionBuilder * Split test in to multiple parts * Use a custom error type if creating joinsplit descriptions fails * detach wallet from miner * fix GetScriptForMining() CReserveKey::keepKey() issue * add CReserveScript to allow modular script keeping/returning - use one CReserveScript per mining thread * miner: rename UpdateRequestCount signal to ResetRequestCount * test: Fetch coinbase address from coinbase UTXOs After upstream PR bitcoin/bitcoin#5994, the first call to getnewaddress after startup does not return the address being used by the miner. * test: Make expected_utxos optional in get_coinbase_address() * Add comments * Move utiltest.cpp from wallet to common This ensures it is accessible by the test suite when the wallet is disabled. * Move payment disclosure code and tests into wallet The code was already compiled as part of the wallet, but the tests were not, meaning that the tests would fail to compile when the wallet was disabled. * depends: Use full path to cargo binary The native binaries generated in the depends system are available on the path, but system binaries are still visible. This change ensures we use cargo from the depends system rather than whatever might be installed locally. * depends: Generalise the rust package cross-compilation functions * depends: Add rust-std hash for aarch64-unknown-linux-gnu Usage on Debian / Ubuntu: > $ sudo apt install g++-aarch64-linux-gnu > $ HOST=aarch64-linux-gnu ./zcutil/build.sh Currently fails to cross-compile due to later configuration issues in the depends system that need to be worked around. * depends: Compile bdb with --disable-atomics on aarch64 This sidesteps the problem where the atomics check tries to run a test binary, which cannot be performed during cross compilation. We should replace this with a better solution in future. Part of #3710. * (testnet) Fall back to hardcoded shielded pool balance to avoid reorgs. * (testnet) Reject blocks that result in turnstile violations * depends: Update .gitignore * configure: Guess -march for libsnark OPTFLAGS instead of hard-coding When cross-compiling, this will remove the -march flag entirely unless the user specifies CONFIGURE_FLAGS="--with-gcc-arch=<arch>". * (testnet/regtest) Avoid mining transactions that would violate the turnstile. * Update nMinimumChainWork using block 497000. * Add checkpoint for block 497000. * Fix tallying for Sprout/Sapling value pools. * Consolidate logic to enable turnstile auditing for testnet/regtest/mainnet. * Use existing chainparams variable * Add newlines to turntile log messages for miner * Check blockhash of fallback block for Sprout value pool balance * Added documentation warnings about DNS rebinding attacks, issue #3841 * Added responsible disclosure statement for issue #3869 * Change SproutValuePoolCheckpointEnabled to ZIP209Activated * Only enforce Sapling turnstile if balance values have been populated. * Update COPYRIGHT_YEAR in clientversion.h to 2019 Update COPYRIGHT_YEAR in clientversion.h to 2019 * Do not enable ZIP209 on regtest right now. * (minor) Remove added newline. * Rename and update comment * (wallet) Check that the commitment matches the note plaintext provided by the sender. * Update release notes for 2.0.4 Co-authored-by: Jack Grigg <jack@z.cash> * Electric Coin Company * Minor speling changes * make-release.py: Versioning changes for 2.0.4-rc1. * make-release.py: Updated manpages for 2.0.4-rc1. * make-release.py: Updated release notes and changelog for 2.0.4-rc1. * Fix typo in release notes. * Update _COPYRIGHT_YEAR in configure.ac to 2019 Update _COPYRIGHT_YEAR in configure.ac to 2019 * fix enable-debug build DB_COINS undefined * Fix proton patch regression. #3916 * Fix OpenSSL reproducible build regression * Patch out proton::url deprecation as workaround for build warnings * make-release.py: Versioning changes for 2.0.4. * make-release.py: Updated manpages for 2.0.4. * make-release.py: Updated release notes and changelog for 2.0.4. * Update company name. * add -addressindex changes for bitcore insight block explorer * tests: adds unit test for IsPayToPublicKeyHash method * Add Blossom to upgrade list * init: Fix new HD seed generation for previously-encrypted wallets Closes #3607. * Creates checklist template for new PRs being opened and addresses Str4d's suggestion for using GitHub handles * Adding addressindex.h to Makefile.am Co-authored by: Daira Hopwood <daira@jacaranda.org> * Add testnet and regtest experimental feature: -developersetpoolsizezero * add -spentindex changes for bitcore insight block explorer * Update boost from v1.69.0 to v1.70.0. #3947 * Add qa test for experimental feature: -developersetpoolsizezero * Enable ZIP209 on mainnet and set fallback Sprout pool balance. * Enable experimental feature -developersetpoolsizezero on mainnet. * Add rpc to enable and disable Sprout to Sapling migration * Move migration logic to ChainTip * Documentation cleanup * Additional locking and race condition prevention * Refactor wait_and_assert_operationid_status to allow returning the result * Set min depth when selecting notes to migrate * add -timestampindex for bitcore insight block explorer * Check for full failure message in test case * Add migration options to conf file Co-authored-by: Simon <simon@bitcartel.com> * remove extra hyphen Co-Authored-By: Eirik0 <eirik@z.cash> * Create method for getting HD seed in RPCs * Add rpc to get Sprout to Sapling migration status * Fix help message * Test migration using both the parameter and the default Sapling address * Fix typos and update documentation * use -valueBalance rather than vpub_new to calculate migrated amount * Do not look at vin/vout when determining migration txs and other cleanup * Calculate the number of confimations in the canonical way * Do not throw an exception if HD Seed is not found when exporting wallet * 3873 z_setmigration cli bool enable arg conversion * make-release.py: Versioning changes for 2.0.5-rc1. * make-release.py: Updated manpages for 2.0.5-rc1. * make-release.py: Updated release notes and changelog for 2.0.5-rc1. * add curl to package list for gitian lxc container * Update chain work and checkpoint using block 525000. * Notable changes for v2.0.5 * Add missing word to release notes * make-release.py: Versioning changes for 2.0.5. * make-release.py: Updated manpages for 2.0.5. * make-release.py: Updated release notes and changelog for 2.0.5. * Correctly account for migration transactions in the mempool Co-authored-by: LarryRuane <larry@z.cash> * Store the migration operation id rather than the operation iteslf * Rename variable and add comment * Notable changes for v2.0.5-1 * Fix summing available funds * Add the amount migrated to the operation's result * coinsView is required when using TransactionBuilder if there may be Sprout change * make-release.py: Versioning changes for 2.0.5-1. * make-release.py: Updated manpages for 2.0.5-1. * make-release.py: Updated release notes and changelog for 2.0.5-1. * Remove unused specifier from format string. The extra specifier meant that a runtime error would be thrown during Sprout to Sapling migration if `zrpcunsafe` logging was enabled: "tinyformat: Too many conversion specifiers in format string" * Don't allow migration when node is syncing at launch or after waking up. * Generalize TransactionBuilder and CreateNewContextualCMutableTransaction to allow choosing the expiry delta. Signed-off-by: Daira Hopwood <daira@jacaranda.org> * Repair calls to TransactionBuilder from tests. Signed-off-by: Daira Hopwood <daira@jacaranda.org> * Change expiry delta for migration transactions to 450 blocks. Signed-off-by: Daira Hopwood <daira@jacaranda.org> * Test the expiry height of migration transactions. Signed-off-by: Daira Hopwood <daira@jacaranda.org> * Fix cosmetic spacing issue in z_setmigration help. Signed-off-by: Daira Hopwood <daira@jacaranda.org> * Do not automatically remove async migration operations and return txids * Add logging for Sprout to Sapling migration transaction generation * Fix LogPrint statements * Notable changes for v2.0.5-2 * Release notes wording and punctuation * make-release.py: Versioning changes for 2.0.5-2. * make-release.py: Updated manpages for 2.0.5-2. * make-release.py: Updated release notes and changelog for 2.0.5-2. * Update ZIP reference * Set BitcoinZ version to 2.0.5 * Fixed merge bug Let's use the chainparams passed into the function rather than the static class instance. * Support to reduce future block time window to 5 mins Check timestamp is within the allowed 5 minute window to help decrease effectiveness of timewarp attacks * Added reorg protection This commit implements some basic reorg protection by implementing a rolling 10 block checkpoint. This limits the number of blocks that a miner can mine in secret to a maximum of 10. * Don't require flag to enable reorg projection * Add disablereorgprotection flag Use the -disablereorgprotection flag to disable reorg protection. * Add checkpoints * Zeuz Fingerz release to go live on block 364400 * ignore shutdown process when reorg > 100 blocks * Set rolling checkpoint height to 6 blocks * Set minimum protocol version to 77007
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
…_target_size b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe) Pull request description: Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
No description provided.