[rpc] Add initialblockdownload to getblockchaininfo #11258

jnewbery · 2017-09-06T18:14:39Z

Exposing whether the node is in IBD would help for testing, and may be useful in general, particularly for developers.

First discussed in #10357 here: #10357 (review)

... we could simplify this (and possibly other) tests by just adding a way to know if a node is in IBD. I'd like to do that, but I'm not sure it makes sense to complicate this PR with discussion over how that information should be made available. Eg it's not clear to me that the notion of being in IBD is worth exposing to the casual user, versus a hidden rpc call or something, since the definition has changed over time, and may continue to change in the future. But I still do agree that at least for testing purposes it would be far simpler to expose the field somehow...

This PR currently implements the simplest way of doing this: adding an initialblockdownload field to getblockchaininfo. Other approaches we could take:

add a new debug RPC method that exposes IBD and potentially other information.
add a parameter to getblockchaininfo, eg debug_info, which would cause it to return debug information including IBD
add a query string to the url ?debug=true which would cause RPCs to return additional debug information.

I quite like the idea of (3). Feedback on these and other approaches very much welcomed!

@sdaftuar @laanwj

laanwj · 2017-09-06T20:36:58Z

Concept ACK

add a query string to the url ?debug=true which would cause RPCs to return additional debug information.

I don't like adding query string arguments to the RPC mechanism. This would add yet another way of passing in arguments, which is confusing, IMO. If this is to be made optional, adding an optional debug argument to get*info seems to be a better way, as it works within the system.

TheBlueMatt · 2017-09-06T20:50:44Z

utACK eb45b179abe2a79a990fb8c0a437654e8aa6b653

maflcko · 2017-09-06T20:54:38Z

src/rpc/blockchain.cpp

@@ -1131,6 +1131,7 @@ UniValue getblockchaininfo(const JSONRPCRequest& request)
            "  \"difficulty\": xxxxxx,     (numeric) the current difficulty\n"
            "  \"mediantime\": xxxxxx,     (numeric) median time for the current best block\n"
            "  \"verificationprogress\": xxxx, (numeric) estimate of verification progress [0..1]\n"
+            "  \"initialblockdownload\": xxxx, (bool) is this node in Initial Block Download mode\n"


Maybe just mention that the meaning may change any time and it is only meant for debugging?

I think it is useful in general.

I agree with @MarcoFalke, even though it might be useful in general this should mention that the interpretation (and possibly even use) of this is internal and subject to change.

promag

Concept ACK.

promag · 2017-09-06T20:58:20Z

src/rpc/blockchain.cpp

@@ -1131,6 +1131,7 @@ UniValue getblockchaininfo(const JSONRPCRequest& request)
            "  \"difficulty\": xxxxxx,     (numeric) the current difficulty\n"
            "  \"mediantime\": xxxxxx,     (numeric) median time for the current best block\n"
            "  \"verificationprogress\": xxxx, (numeric) estimate of verification progress [0..1]\n"
+            "  \"initialblockdownload\": xxxx, (bool) is this node in Initial Block Download mode\n"


s/mode/state?

Nit, align descriptions 🙄.

promag · 2017-09-06T21:01:22Z

test/functional/blockchain.py

+        assert_equal(res["blocks"], 200)
+        assert_equal(res["headers"], 200)
+        assert_equal(len(res["bestblockhash"]), 64)
+        assert "difficulty" in res


Instead of multiple assert str in res, how about assert_equal(res.keys(), ...)? This way if someone adds a new response field or changes the response, it must update this tests too.

So only assert known values, and exact keys.

promag · 2017-09-06T21:16:54Z

Another approach is to create the generic read-only calls:

liststates returns an array of state names that can be queried;
querystate state_name returns the state value;
waitstate state_name (timeout) waits for state change or timeout and returns the state value;

In this case: bitcoin-cli querystate initialblockdownload.

Well, just an approach as @jnewbery asked 😄.

laanwj · 2017-09-06T21:30:51Z

@promag The idea is ok, though one problem is that that creates cross-cutting RPC calls, whereas we've been moving to subsystem-specific ones for a while (this was one of the reasons to get rid of the generic getinfo).

promag · 2017-09-06T23:16:21Z

one problem is that that creates cross-cutting RPC calls

I don't agree with that, it only forces the same RPC API for all states, but each subsystem deals with it's states, it's not like getinfo which is a mix of data.

So there would be a middleware where each subsystem must declare the available states and the RPC handler just delegates the state evaluation there.

In other words, I also disagree with having fat RPC's, but I agree with one horizontal API. For instance, the config is horizontal for all subsystems.

gmaxwell · 2017-09-08T21:30:08Z

Concept ACK but I'm somewhat concerned that IsInitialBlockDownload means something that is not equal to what the user means by "in initial block download". A node is unable to know if its caught up or not, and this is a somewhat reliable heuristic. But for example, if you have a node partitioned off on some fork or something it may well return false here. The concern might be addressable by just throwing the word "estimated" or similar in there somewhere.

morcos · 2017-09-11T18:31:49Z

ACK eb45b17

I agree slightly better to add some warning to the description of IBD, but that's a nit as far as I'm concerned

jnewbery · 2017-09-12T15:09:55Z

Added wording "The use and interpretation of this property may change between releases."
added whitespace-only commit to fix up alignment for help text
rebased on master

maflcko · 2017-09-12T15:36:02Z

Please remove the [WIP], when ready for review/merge

jnewbery · 2017-09-12T16:33:13Z

Ready for review/merge. PR title updated.

Travis failed for The command "if [ -d ~/.bitcoin ]; then false; fi" exited with 1. Does the cache need to be cleared?

sipa · 2017-09-12T16:40:05Z

@jnewbery Rebase to fix that.

maflcko · 2017-09-12T16:41:33Z

This was already rebased and we have the rm -rf in travis' yaml. I think the issue needs more investigation. (Might want to revert from the travis' yaml temporarily).

jnewbery · 2017-10-26T15:58:34Z

Rebased.

I've updated the help text to:

"initialblockdownload": xxxx, (bool) (debug information) estimate of whether this node is in Initial Block Download mode.

to address Greg's comment: #11258 (comment)

No functional changes.

I think this is ready for merge.

jnewbery · 2017-10-26T16:12:15Z

Silent rebase conflict. Should be fixed now

luke-jr · 2017-11-10T09:14:54Z

The current commits only add comments to the functional test (claiming it tests it), but doesn't actually add any tests...?

jnewbery · 2017-11-10T17:53:28Z

The current commits only add comments to the functional test (claiming it tests it), but doesn't actually add any tests...?

https://github.com/bitcoin/bitcoin/pull/11258/files#diff-31748911f612ce1d09bc82d04f452592R62 adds initialblockdownload to the expected keys returned in getblockchaininfo

promag · 2017-11-10T18:45:26Z

Unless @luke-jr expects a test for the value.

jnewbery · 2017-11-10T20:14:00Z

Unless @luke-jr expects a test for the value.

Almost none of the fields from getblockchaininfo are tested for value. PRs welcome to change that.

It's slightly frustrating that this is a minimal functionality PR that has been open for 2 months, ACK'ed by several people, rebased many times, and is now getting nitted because the test coverage for the new functionality doesn't exceed the test coverage for existing functionality. The fact that we're getting down to these micronits suggests it's time to merge this PR (or abandon it).

achow101 · 2017-11-10T20:16:37Z

utACK 1141364

@sdaftuar

1141364 [trivial] (whitespace only) fix getblockchaininfo alignment (John Newbery) bd9c181 [rpc] Add initialblockdownload to getblockchaininfo (John Newbery) Pull request description: Exposing whether the node is in IBD would help for testing, and may be useful in general, particularly for developers. First discussed in #10357 here: #10357 (review) > ... we could simplify this (and possibly other) tests by just adding a way to know if a node is in IBD. I'd like to do that, but I'm not sure it makes sense to complicate this PR with discussion over how that information should be made available. Eg it's not clear to me that the notion of being in IBD is worth exposing to the casual user, versus a hidden rpc call or something, since the definition has changed over time, and may continue to change in the future. But I still do agree that at least for testing purposes it would be far simpler to expose the field somehow... This PR currently implements the simplest way of doing this: adding an `initialblockdownload` field to `getblockchaininfo`. Other approaches we could take: 1. add a new debug RPC method that exposes `IBD` and potentially other information. 2. add a parameter to `getblockchaininfo`, eg `debug_info`, which would cause it to return debug information including IBD 3. add a query string to the url `?debug=true` which would cause RPCs to return additional debug information. I quite like the idea of (3). Feedback on these and other approaches very much welcomed! @sdaftuar @laanwj Tree-SHA512: a6dedd47f8c9bd38769cc597524466250041136feb33500644b9c48d0ffe4e3eeeb2587b5bbc6420364ebdd2667df807fbb50416f9a7913bbf11a14ea86dc0d4

Github-Pull: bitcoin#11258 Rebased-From: bd9c181

@sdaftuar

1141364 [trivial] (whitespace only) fix getblockchaininfo alignment (John Newbery) bd9c181 [rpc] Add initialblockdownload to getblockchaininfo (John Newbery) Pull request description: Exposing whether the node is in IBD would help for testing, and may be useful in general, particularly for developers. First discussed in bitcoin#10357 here: bitcoin#10357 (review) > ... we could simplify this (and possibly other) tests by just adding a way to know if a node is in IBD. I'd like to do that, but I'm not sure it makes sense to complicate this PR with discussion over how that information should be made available. Eg it's not clear to me that the notion of being in IBD is worth exposing to the casual user, versus a hidden rpc call or something, since the definition has changed over time, and may continue to change in the future. But I still do agree that at least for testing purposes it would be far simpler to expose the field somehow... This PR currently implements the simplest way of doing this: adding an `initialblockdownload` field to `getblockchaininfo`. Other approaches we could take: 1. add a new debug RPC method that exposes `IBD` and potentially other information. 2. add a parameter to `getblockchaininfo`, eg `debug_info`, which would cause it to return debug information including IBD 3. add a query string to the url `?debug=true` which would cause RPCs to return additional debug information. I quite like the idea of (3). Feedback on these and other approaches very much welcomed! @sdaftuar @laanwj Tree-SHA512: a6dedd47f8c9bd38769cc597524466250041136feb33500644b9c48d0ffe4e3eeeb2587b5bbc6420364ebdd2667df807fbb50416f9a7913bbf11a14ea86dc0d4

@sdaftuar

1141364 [trivial] (whitespace only) fix getblockchaininfo alignment (John Newbery) bd9c181 [rpc] Add initialblockdownload to getblockchaininfo (John Newbery) Pull request description: Exposing whether the node is in IBD would help for testing, and may be useful in general, particularly for developers. First discussed in bitcoin#10357 here: bitcoin#10357 (review) > ... we could simplify this (and possibly other) tests by just adding a way to know if a node is in IBD. I'd like to do that, but I'm not sure it makes sense to complicate this PR with discussion over how that information should be made available. Eg it's not clear to me that the notion of being in IBD is worth exposing to the casual user, versus a hidden rpc call or something, since the definition has changed over time, and may continue to change in the future. But I still do agree that at least for testing purposes it would be far simpler to expose the field somehow... This PR currently implements the simplest way of doing this: adding an `initialblockdownload` field to `getblockchaininfo`. Other approaches we could take: 1. add a new debug RPC method that exposes `IBD` and potentially other information. 2. add a parameter to `getblockchaininfo`, eg `debug_info`, which would cause it to return debug information including IBD 3. add a query string to the url `?debug=true` which would cause RPCs to return additional debug information. I quite like the idea of (3). Feedback on these and other approaches very much welcomed! @sdaftuar @laanwj Tree-SHA512: a6dedd47f8c9bd38769cc597524466250041136feb33500644b9c48d0ffe4e3eeeb2587b5bbc6420364ebdd2667df807fbb50416f9a7913bbf11a14ea86dc0d4

@sdaftuar

1141364 [trivial] (whitespace only) fix getblockchaininfo alignment (John Newbery) bd9c181 [rpc] Add initialblockdownload to getblockchaininfo (John Newbery) Pull request description: Exposing whether the node is in IBD would help for testing, and may be useful in general, particularly for developers. First discussed in bitcoin#10357 here: bitcoin#10357 (review) > ... we could simplify this (and possibly other) tests by just adding a way to know if a node is in IBD. I'd like to do that, but I'm not sure it makes sense to complicate this PR with discussion over how that information should be made available. Eg it's not clear to me that the notion of being in IBD is worth exposing to the casual user, versus a hidden rpc call or something, since the definition has changed over time, and may continue to change in the future. But I still do agree that at least for testing purposes it would be far simpler to expose the field somehow... This PR currently implements the simplest way of doing this: adding an `initialblockdownload` field to `getblockchaininfo`. Other approaches we could take: 1. add a new debug RPC method that exposes `IBD` and potentially other information. 2. add a parameter to `getblockchaininfo`, eg `debug_info`, which would cause it to return debug information including IBD 3. add a query string to the url `?debug=true` which would cause RPCs to return additional debug information. I quite like the idea of (3). Feedback on these and other approaches very much welcomed! @sdaftuar @laanwj Tree-SHA512: a6dedd47f8c9bd38769cc597524466250041136feb33500644b9c48d0ffe4e3eeeb2587b5bbc6420364ebdd2667df807fbb50416f9a7913bbf11a14ea86dc0d4

@sdaftuar

1141364 [trivial] (whitespace only) fix getblockchaininfo alignment (John Newbery) bd9c181 [rpc] Add initialblockdownload to getblockchaininfo (John Newbery) Pull request description: Exposing whether the node is in IBD would help for testing, and may be useful in general, particularly for developers. First discussed in bitcoin#10357 here: bitcoin#10357 (review) > ... we could simplify this (and possibly other) tests by just adding a way to know if a node is in IBD. I'd like to do that, but I'm not sure it makes sense to complicate this PR with discussion over how that information should be made available. Eg it's not clear to me that the notion of being in IBD is worth exposing to the casual user, versus a hidden rpc call or something, since the definition has changed over time, and may continue to change in the future. But I still do agree that at least for testing purposes it would be far simpler to expose the field somehow... This PR currently implements the simplest way of doing this: adding an `initialblockdownload` field to `getblockchaininfo`. Other approaches we could take: 1. add a new debug RPC method that exposes `IBD` and potentially other information. 2. add a parameter to `getblockchaininfo`, eg `debug_info`, which would cause it to return debug information including IBD 3. add a query string to the url `?debug=true` which would cause RPCs to return additional debug information. I quite like the idea of (3). Feedback on these and other approaches very much welcomed! @sdaftuar @laanwj Tree-SHA512: a6dedd47f8c9bd38769cc597524466250041136feb33500644b9c48d0ffe4e3eeeb2587b5bbc6420364ebdd2667df807fbb50416f9a7913bbf11a14ea86dc0d4

laanwj added the RPC/REST/ZMQ label Sep 6, 2017

maflcko reviewed Sep 6, 2017

View reviewed changes

promag reviewed Sep 6, 2017

View reviewed changes

jnewbery force-pushed the expose_ibd branch from eb45b17 to afc7f6d Compare September 12, 2017 15:08

jnewbery changed the title ~~[WIP] [rpc] Add initialblockdownload to getblockchaininfo~~ [rpc] Add initialblockdownload to getblockchaininfo Sep 12, 2017

jnewbery force-pushed the expose_ibd branch from afc7f6d to c80d5a3 Compare October 26, 2017 15:57

jnewbery added 2 commits October 26, 2017 12:11

[rpc] Add initialblockdownload to getblockchaininfo

bd9c181

[trivial] (whitespace only) fix getblockchaininfo alignment

1141364

jnewbery force-pushed the expose_ibd branch from c80d5a3 to 1141364 Compare October 26, 2017 16:11

sipa merged commit 1141364 into bitcoin:master Nov 11, 2017

jnewbery deleted the expose_ibd branch November 11, 2017 00:16

luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Nov 11, 2017

[rpc] Add initialblockdownload to getblockchaininfo

4007c82

Github-Pull: bitcoin#11258 Rebased-From: bd9c181

meshcollider mentioned this pull request Jan 17, 2018

TODO for release notes 0.16.0 #11054

Closed

This was referenced Dec 31, 2018

getblockchaininfo - verificationprogress never goes 100% dogecoin/dogecoin#1547

Closed

[rpc] Add initialblockdownload to getblockchaininfo dogecoin/dogecoin#1560

Closed

langerhans mentioned this pull request Dec 31, 2018

Fix verification progress dogecoin/dogecoin#1561

Merged

dagurval mentioned this pull request Feb 22, 2019

[rpc] add initialblockdownload to getblockchaininfo bcoin-org/bcash#131

Open

dagurval mentioned this pull request Mar 14, 2019

[rpc] Add initialblockdownload to getblockchaininfo BitcoinUnlimited/BitcoinUnlimited#1606

Merged

Dolaned mentioned this pull request May 6, 2019

inintialblockdownload project-ecc/eccoin#172

Closed

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[rpc] Add initialblockdownload to getblockchaininfo #11258

[rpc] Add initialblockdownload to getblockchaininfo #11258

jnewbery commented Sep 6, 2017

laanwj commented Sep 6, 2017

TheBlueMatt commented Sep 6, 2017

maflcko Sep 6, 2017

promag Sep 6, 2017

laanwj Sep 6, 2017

promag left a comment

promag Sep 6, 2017

promag Sep 6, 2017

promag Sep 6, 2017

promag commented Sep 6, 2017

laanwj commented Sep 6, 2017

promag commented Sep 6, 2017

gmaxwell commented Sep 8, 2017

morcos commented Sep 11, 2017

jnewbery commented Sep 12, 2017

maflcko commented Sep 12, 2017

jnewbery commented Sep 12, 2017

sipa commented Sep 12, 2017

maflcko commented Sep 12, 2017

jnewbery commented Oct 26, 2017

jnewbery commented Oct 26, 2017

luke-jr commented Nov 10, 2017

jnewbery commented Nov 10, 2017

promag commented Nov 10, 2017

jnewbery commented Nov 10, 2017

achow101 commented Nov 10, 2017

[rpc] Add initialblockdownload to getblockchaininfo #11258

[rpc] Add initialblockdownload to getblockchaininfo #11258

Conversation

jnewbery commented Sep 6, 2017

laanwj commented Sep 6, 2017

TheBlueMatt commented Sep 6, 2017

maflcko Sep 6, 2017

Choose a reason for hiding this comment

promag Sep 6, 2017

Choose a reason for hiding this comment

laanwj Sep 6, 2017

Choose a reason for hiding this comment

promag left a comment

Choose a reason for hiding this comment

promag Sep 6, 2017

Choose a reason for hiding this comment

promag Sep 6, 2017

Choose a reason for hiding this comment

promag Sep 6, 2017

Choose a reason for hiding this comment

promag commented Sep 6, 2017

laanwj commented Sep 6, 2017

promag commented Sep 6, 2017

gmaxwell commented Sep 8, 2017

morcos commented Sep 11, 2017

jnewbery commented Sep 12, 2017

maflcko commented Sep 12, 2017

jnewbery commented Sep 12, 2017

sipa commented Sep 12, 2017

maflcko commented Sep 12, 2017

jnewbery commented Oct 26, 2017

jnewbery commented Oct 26, 2017

luke-jr commented Nov 10, 2017

jnewbery commented Nov 10, 2017

promag commented Nov 10, 2017

jnewbery commented Nov 10, 2017

achow101 commented Nov 10, 2017