New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rpc] getblockchaininfo: add size_on_disk, prune_target_size #11367

Merged
merged 1 commit into from Oct 9, 2017

Conversation

Projects
None yet
10 participants
@esotericnonsense
Contributor

esotericnonsense commented Sep 19, 2017

No description provided.

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 19, 2017

Contributor

If #11359 gets in it might be nice to have the prune hwm in there too, but I didn't want to add the dependency.

Contributor

esotericnonsense commented Sep 19, 2017

If #11359 gets in it might be nice to have the prune hwm in there too, but I didn't want to add the dependency.

@jnewbery

concept ACK. One nit inline.

Consider adding an explicit test for the getblockchaininfo RPC method to test/functional/blockchain.py.

Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
@promag

This comment has been minimized.

Show comment
Hide comment
@promag

promag Sep 20, 2017

Member

Test for getblockchaininfo added in #11370.

Member

promag commented Sep 20, 2017

Test for getblockchaininfo added in #11370.

@esotericnonsense esotericnonsense changed the title from RPC: getblockchaininfo: Add disk_size, prune_target_size to [rpc] getblockchaininfo: add size_on_disk, prune_target_size Sep 20, 2017

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 20, 2017

Contributor

A few mishaps with the commit message there, sorry.

I have merged this with #11366 and closed #11366.

Contributor

esotericnonsense commented Sep 20, 2017

A few mishaps with the commit message there, sorry.

I have merged this with #11366 and closed #11366.

@jnewbery

one more documentation nit. Otherwise looks good to me

Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
Show outdated Hide outdated src/validation.h Outdated
Show outdated Hide outdated src/validation.h Outdated
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Sep 22, 2017

Member

Concept ACK

Member

jonasschnelli commented Sep 22, 2017

Concept ACK

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 22, 2017

Contributor

Rebased on master, moved the lock into CalculateCurrentUsage, CBlockIndex *block => CBlockIndex* block.

Travis failed on one platform (iirc linux64) on a timeout in p2p-segwit last time. Not sure why. We'll see how it goes. edit: looks good.

Contributor

esotericnonsense commented Sep 22, 2017

Rebased on master, moved the lock into CalculateCurrentUsage, CBlockIndex *block => CBlockIndex* block.

Travis failed on one platform (iirc linux64) on a timeout in p2p-segwit last time. Not sure why. We'll see how it goes. edit: looks good.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Sep 22, 2017

Member

utACK 071879a

Member

sipa commented Sep 22, 2017

utACK 071879a

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Sep 22, 2017

Member

I've just tested this with -prune=1 and I get the bizarre result:

→ bitcoin-cli getblockchaininfo
{
...
  "prune_target_size": 18446744073709551615,

That's because of this section in init.cpp:

    nPruneTarget = (uint64_t) nPruneArg * 1024 * 1024;
    if (nPruneArg == 1) {  // manual pruning: -prune=1
        LogPrintf("Block pruning enabled.  Use RPC call pruneblockchain(height) to manually prune block and undo files.\n");
        nPruneTarget = std::numeric_limits<uint64_t>::max();
        fPruneMode = true;
...

Returning the max int for prune_target_size is meaningless and confusing. I recommend you don't return prune_target_size if bitcoind is configured for manual pruning (ie if -prune=1) (and perhaps also add another field pruning_mode that returns whether pruning is automatic or manual).

Member

jnewbery commented Sep 22, 2017

I've just tested this with -prune=1 and I get the bizarre result:

→ bitcoin-cli getblockchaininfo
{
...
  "prune_target_size": 18446744073709551615,

That's because of this section in init.cpp:

    nPruneTarget = (uint64_t) nPruneArg * 1024 * 1024;
    if (nPruneArg == 1) {  // manual pruning: -prune=1
        LogPrintf("Block pruning enabled.  Use RPC call pruneblockchain(height) to manually prune block and undo files.\n");
        nPruneTarget = std::numeric_limits<uint64_t>::max();
        fPruneMode = true;
...

Returning the max int for prune_target_size is meaningless and confusing. I recommend you don't return prune_target_size if bitcoind is configured for manual pruning (ie if -prune=1) (and perhaps also add another field pruning_mode that returns whether pruning is automatic or manual).

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 22, 2017

Contributor

Hmmm. Yes, that isn't desirable behaviour.

Options I can see:
a) nPruneArg is exposed from init.cpp, or an fPruneManualMode (in which case it could be explicitly checked for in FindFilesToPrune)
or
b) the RPC checks for int_max to determine whether automatic pruning is disabled

b) fixes the RPC without having to touch pruning code but otherwise seems a bit messy to me.

Contributor

esotericnonsense commented Sep 22, 2017

Hmmm. Yes, that isn't desirable behaviour.

Options I can see:
a) nPruneArg is exposed from init.cpp, or an fPruneManualMode (in which case it could be explicitly checked for in FindFilesToPrune)
or
b) the RPC checks for int_max to determine whether automatic pruning is disabled

b) fixes the RPC without having to touch pruning code but otherwise seems a bit messy to me.

@MeshCollider

This comment has been minimized.

Show comment
Hide comment
@MeshCollider

MeshCollider Sep 23, 2017

Member

Or instead of b) just see if gArgs.GetArg("-prune", 0) is 1?
But yeah maybe adding a manual prune bool to validation.cpp would be more sensible

Member

MeshCollider commented Sep 23, 2017

Or instead of b) just see if gArgs.GetArg("-prune", 0) is 1?
But yeah maybe adding a manual prune bool to validation.cpp would be more sensible

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Sep 23, 2017

Member

I think just testing the value of gArgs.GetArg("-prune", 0) is appropriate for this pr. No need to touch the init or validation code.

Member

jnewbery commented Sep 23, 2017

I think just testing the value of gArgs.GetArg("-prune", 0) is appropriate for this pr. No need to touch the init or validation code.

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 24, 2017

Contributor
$ grep "prune" ~/.bitcoin/bitcoin.conf
prune=550
$ ./bitcoin-cli getblockchaininfo | grep "prune"
  "pruned": true,
  "pruneheight": 0,
  "pruneauto": true,
  "prune_target_size": 576716800,

$ grep "prune" ~/.bitcoin/bitcoin.conf
prune=1
$ ./bitcoin-cli getblockchaininfo | grep "prune"
  "pruned": true,
  "pruneheight": 0,
  "pruneauto": false,

$ grep "prune" ~/.bitcoin/bitcoin.conf
prune=0
$ ./bitcoin-cli getblockchaininfo | grep "prune"
  "pruned": false,
Contributor

esotericnonsense commented Sep 24, 2017

$ grep "prune" ~/.bitcoin/bitcoin.conf
prune=550
$ ./bitcoin-cli getblockchaininfo | grep "prune"
  "pruned": true,
  "pruneheight": 0,
  "pruneauto": true,
  "prune_target_size": 576716800,

$ grep "prune" ~/.bitcoin/bitcoin.conf
prune=1
$ ./bitcoin-cli getblockchaininfo | grep "prune"
  "pruned": true,
  "pruneheight": 0,
  "pruneauto": false,

$ grep "prune" ~/.bitcoin/bitcoin.conf
prune=0
$ ./bitcoin-cli getblockchaininfo | grep "prune"
  "pruned": false,
@promag

This comment has been minimized.

Show comment
Hide comment
@promag

promag Sep 24, 2017

Member

Please rebase with #11370.

Member

promag commented Sep 24, 2017

Please rebase with #11370.

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 25, 2017

Contributor

Rebased on master and added tests for the relevant fields in test/functional/blockchain.py.

My box:

...
blockchain.py                  | ✓ Passed  | 13 s
...
ALL                            | ✓ Passed  | 695 s (accumulated) 
Runtime: 180 s
Contributor

esotericnonsense commented Sep 25, 2017

Rebased on master and added tests for the relevant fields in test/functional/blockchain.py.

My box:

...
blockchain.py                  | ✓ Passed  | 13 s
...
ALL                            | ✓ Passed  | 695 s (accumulated) 
Runtime: 180 s
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
@jnewbery

Implementation looks good. I've slightly tested and it works well.

A few style nits inline.

Show outdated Hide outdated test/functional/blockchain.py Outdated
Show outdated Hide outdated test/functional/blockchain.py Outdated
Show outdated Hide outdated test/functional/blockchain.py Outdated
Show outdated Hide outdated src/rpc/blockchain.cpp Outdated
@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 26, 2017

Contributor

Changes made and rebased on master (second push, first was a mistake).

Contributor

esotericnonsense commented Sep 26, 2017

Changes made and rebased on master (second push, first was a mistake).

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Sep 26, 2017

Member

Tested ACK 2b73ece

Member

jnewbery commented Sep 26, 2017

Tested ACK 2b73ece

{
LOCK(cs_LastBlockFile);

This comment has been minimized.

@TheBlueMatt

TheBlueMatt Sep 27, 2017

Contributor

While you're at it can you add this lock to PruneOneBlockFile and GetBlockFileInfo (both of which are only called without lock during testing, so its not an actual issue, just annoying).

@TheBlueMatt

TheBlueMatt Sep 27, 2017

Contributor

While you're at it can you add this lock to PruneOneBlockFile and GetBlockFileInfo (both of which are only called without lock during testing, so its not an actual issue, just annoying).

This comment has been minimized.

@esotericnonsense

esotericnonsense Sep 29, 2017

Contributor

Done in b7dfc6c.

@esotericnonsense

esotericnonsense Sep 29, 2017

Contributor

Done in b7dfc6c.

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Sep 29, 2017

Member

Needs rebase™

Member

MarcoFalke commented Sep 29, 2017

Needs rebase™

[rpc] getblockchaininfo: add size_on_disk, prune_target_size, automat…
…ic_pruning

Fix pruneheight help text.
Move fPruneMode block to match output ordering with help text.
Add functional tests for new fields in getblockchaininfo.
@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Sep 29, 2017

Contributor

®ebased

Contributor

esotericnonsense commented Sep 29, 2017

®ebased

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke
Member

MarcoFalke commented Sep 30, 2017

utA©K b7dfc6c

@TheBlueMatt

This comment has been minimized.

Show comment
Hide comment
@TheBlueMatt

TheBlueMatt Oct 2, 2017

Contributor

re-utACK b7dfc6c. In the future, can you avoid rebasing when squashing unless you need to? Avoiding rebasing onto latest master makes it easier for reviewers to identify what changed between reviews.

Contributor

TheBlueMatt commented Oct 2, 2017

re-utACK b7dfc6c. In the future, can you avoid rebasing when squashing unless you need to? Avoiding rebasing onto latest master makes it easier for reviewers to identify what changed between reviews.

@esotericnonsense

This comment has been minimized.

Show comment
Hide comment
@esotericnonsense

esotericnonsense Oct 2, 2017

Contributor

Yes, I can do that in the future. I hadn't considered that.
Do you prefer for each change to be made in an individual commit (with a squash before the PR is implemented)?

Contributor

esotericnonsense commented Oct 2, 2017

Yes, I can do that in the future. I hadn't considered that.
Do you prefer for each change to be made in an individual commit (with a squash before the PR is implemented)?

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Oct 3, 2017

Member
Member

MarcoFalke commented Oct 3, 2017

@laanwj laanwj merged commit b7dfc6c into bitcoin:master Oct 9, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Oct 9, 2017

Merge #11367: [rpc] getblockchaininfo: add size_on_disk, prune_target…
…_size

b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe)

Pull request description:

Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 11, 2017

[rpc] getblockchaininfo: add size_on_disk, prune_target_size, automat…
…ic_pruning

Fix pruneheight help text.
Move fPruneMode block to match output ordering with help text.
Add functional tests for new fields in getblockchaininfo.

Github-Pull: bitcoin#11367
Rebased-From: b7dfc6c

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 11, 2017

[rpc] getblockchaininfo: add size_on_disk, prune_target_size, automat…
…ic_pruning

Fix pruneheight help text.
Move fPruneMode block to match output ordering with help text.
Add functional tests for new fields in getblockchaininfo.

Github-Pull: bitcoin#11367
Rebased-From: b7dfc6c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment