New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle getinfo in bitcoin-cli w/ -getinfo (revival of #8843) #10871

Merged
merged 2 commits into from Sep 28, 2017

Conversation

Projects
None yet
8 participants
@achow101
Member

achow101 commented Jul 18, 2017

Since @laanwj doesn't want to maintain these changes anymore, I will.

This PR is a revival of #8843. I have addressed @jnewbery's comments.

Regarding atomicity, I don't think that is a concern here. This is explicitly a new API and those who use it will know that this is different and that it is not atomic.

@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jul 19, 2017

Member

Concept ACK.
Can you add a tool test for that?

Member

jonasschnelli commented Jul 19, 2017

Concept ACK.
Can you add a tool test for that?

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jul 20, 2017

Member

Can you add a tool test for that?

A simple tool-test like -tx's wont work here as it requires a running bitcoind.
Better to wait for bitcoin-cli testing infrastructure to be in place I guess (#10798)?

Member

laanwj commented Jul 20, 2017

Can you add a tool test for that?

A simple tool-test like -tx's wont work here as it requires a running bitcoind.
Better to wait for bitcoin-cli testing infrastructure to be in place I guess (#10798)?

@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Aug 15, 2017

Member

rebased

Member

achow101 commented Aug 15, 2017

rebased

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Aug 15, 2017

Member

bad rebase:

bitcoin-cli.cpp: In function ‘int CommandLineRPC(int, char**)’:
bitcoin-cli.cpp:393:41: error: ‘GetBoolArg’ was not declared in this scope
         if (GetBoolArg("-getinfo", false)) {
                                         ^

#10798 is now rebased and ready for merge. It should be fairly straightforward to rebase this on top of that to add a functional test.

Member

jnewbery commented Aug 15, 2017

bad rebase:

bitcoin-cli.cpp: In function ‘int CommandLineRPC(int, char**)’:
bitcoin-cli.cpp:393:41: error: ‘GetBoolArg’ was not declared in this scope
         if (GetBoolArg("-getinfo", false)) {
                                         ^

#10798 is now rebased and ready for merge. It should be fairly straightforward to rebase this on top of that to add a functional test.

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Aug 23, 2017

Member

Test here if you want it: https://github.com/jnewbery/bitcoin/tree/pr10871_test

Builds on top of #10798. Test currently fails because the output for bitcoin-cli getinfo and bitcoin-cli -getinfo is different in keypoolsize and unlocked_until. See #8843 (comment) for information about keypoolsize discrepancy.

Member

jnewbery commented Aug 23, 2017

Test here if you want it: https://github.com/jnewbery/bitcoin/tree/pr10871_test

Builds on top of #10798. Test currently fails because the output for bitcoin-cli getinfo and bitcoin-cli -getinfo is different in keypoolsize and unlocked_until. See #8843 (comment) for information about keypoolsize discrepancy.

@TheBlueMatt

This comment has been minimized.

Show comment
Hide comment
@TheBlueMatt

TheBlueMatt Aug 23, 2017

Contributor

I think the docs should at least mention something about atomicity. Every other API to Bitcoin Core is atomic, so it feels very strange to add a new one which is not.

Contributor

TheBlueMatt commented Aug 23, 2017

I think the docs should at least mention something about atomicity. Every other API to Bitcoin Core is atomic, so it feels very strange to add a new one which is not.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Aug 23, 2017

Member

I think the docs should at least mention something about atomicity. Every other API to Bitcoin Core is atomic, so it feels very strange to add a new one which is not.

I know you don't like this particular changej, but this is reaching FUD levels here, what exactly are you concerned about happening here? -getinfo pulls the following information:

getnetworkinfo:
"version"
"protocolversion"
"timeoffset"
"connections"
"proxy"
"relayfee"
"errors"

getblockchaininfo:
"blocks"
"difficulty"
"testnet"

getwalletinfo:
"walletversion"
"balance"
"keypoololdest"
"keypoolsize"
"unlocked_until"
"paytxfee"

Within a group, the values are as atomic as ever. Some of these don't change at runtime at all, some hardly change.

Can you indicate exactly between which two you're afraid non-atomicity is going to give a problem, in practice?

If not, let's please leave this behind us.

Member

laanwj commented Aug 23, 2017

I think the docs should at least mention something about atomicity. Every other API to Bitcoin Core is atomic, so it feels very strange to add a new one which is not.

I know you don't like this particular changej, but this is reaching FUD levels here, what exactly are you concerned about happening here? -getinfo pulls the following information:

getnetworkinfo:
"version"
"protocolversion"
"timeoffset"
"connections"
"proxy"
"relayfee"
"errors"

getblockchaininfo:
"blocks"
"difficulty"
"testnet"

getwalletinfo:
"walletversion"
"balance"
"keypoololdest"
"keypoolsize"
"unlocked_until"
"paytxfee"

Within a group, the values are as atomic as ever. Some of these don't change at runtime at all, some hardly change.

Can you indicate exactly between which two you're afraid non-atomicity is going to give a problem, in practice?

If not, let's please leave this behind us.

@TheBlueMatt

This comment has been minimized.

Show comment
Hide comment
@TheBlueMatt

TheBlueMatt Aug 24, 2017

Contributor

My only real atomicity concern is between blocks and balance - ie that you may call getinfo, see a balance and assume that it is current as of a given block height. It may be of lesser concern in just bitcoin-cli, as its not intended to be used in scripts, hence my note that I'd be ok with just documenting this behavior, though I could still see someone fucking it up in some way.

Contributor

TheBlueMatt commented Aug 24, 2017

My only real atomicity concern is between blocks and balance - ie that you may call getinfo, see a balance and assume that it is current as of a given block height. It may be of lesser concern in just bitcoin-cli, as its not intended to be used in scripts, hence my note that I'd be ok with just documenting this behavior, though I could still see someone fucking it up in some way.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Aug 24, 2017

Member

My only real atomicity concern is between blocks and balance - ie that you may call getinfo, see a balance and assume that it is current as of a given block height.

Yes, that's a valid concern. Also outside this PR - there is currently no way to query up to which height the wallet has processed. Especially when the wallet and node are decoupled further w/ threads or even processes. We should add a curheight or such to getwalletinfo.

Member

laanwj commented Aug 24, 2017

My only real atomicity concern is between blocks and balance - ie that you may call getinfo, see a balance and assume that it is current as of a given block height.

Yes, that's a valid concern. Also outside this PR - there is currently no way to query up to which height the wallet has processed. Especially when the wallet and node are decoupled further w/ threads or even processes. We should add a curheight or such to getwalletinfo.

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Aug 24, 2017

Member

there is currently no way to query up to which height the wallet has processed

I believe that getinfo itself could be misleading here. balance could be for a different height than blocks.

There's been discussion of adding the wallet's best block to getwalletinfo on IRC: https://botbot.me/freenode/bitcoin-core-dev/2017-08-11/?msg=89723092&page=2 and I have a branch to do that, which I'm going to PR after #10286

Member

jnewbery commented Aug 24, 2017

there is currently no way to query up to which height the wallet has processed

I believe that getinfo itself could be misleading here. balance could be for a different height than blocks.

There's been discussion of adding the wallet's best block to getwalletinfo on IRC: https://botbot.me/freenode/bitcoin-core-dev/2017-08-11/?msg=89723092&page=2 and I have a branch to do that, which I'm going to PR after #10286

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Sep 1, 2017

Member

Is this still being actively worked on? Next steps would be:

Member

jnewbery commented Sep 1, 2017

Is this still being actively worked on? Next steps would be:

@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 1, 2017

Member

I am still working on this, just got sidetracked with other stuff.

Member

achow101 commented Sep 1, 2017

I am still working on this, just got sidetracked with other stuff.

@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 5, 2017

Member

Took the test and rebased.

fix the discrepencies between getinfo and -getinfo

What discrepancies?

Member

achow101 commented Sep 5, 2017

Took the test and rebased.

fix the discrepencies between getinfo and -getinfo

What discrepancies?

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Sep 5, 2017

Member

What discrepancies?

See above:

Test currently fails because the output for bitcoin-cli getinfo and bitcoin-cli -getinfo is different in keypoolsize and unlocked_until. See #8843 (comment) for information about keypoolsize discrepancy.

Once #10838 is merged, then that's no longer an issue.

After #10838, I think the way to test this would be to collect the responses from getwalletinfo, getblockchaininfo and getnetworkinfo, merge those dictionaries (hint: https://stackoverflow.com/a/26853961) and then compare that to the response from bitcoin-cli -getinfo.

Member

jnewbery commented Sep 5, 2017

What discrepancies?

See above:

Test currently fails because the output for bitcoin-cli getinfo and bitcoin-cli -getinfo is different in keypoolsize and unlocked_until. See #8843 (comment) for information about keypoolsize discrepancy.

Once #10838 is merged, then that's no longer an issue.

After #10838, I think the way to test this would be to collect the responses from getwalletinfo, getblockchaininfo and getnetworkinfo, merge those dictionaries (hint: https://stackoverflow.com/a/26853961) and then compare that to the response from bitcoin-cli -getinfo.

@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 5, 2017

Member

After #10838, I think the way to test this would be to collect the responses from getwalletinfo, getblockchaininfo and getnetworkinfo, merge those dictionaries (hint: https://stackoverflow.com/a/26853961) and then compare that to the response from bitcoin-cli -getinfo.

I think that we should be doing that instead of comparing against getinfo. This is explicitly a new thing which is similar to getinfo but not exactly replicating it.

Member

achow101 commented Sep 5, 2017

After #10838, I think the way to test this would be to collect the responses from getwalletinfo, getblockchaininfo and getnetworkinfo, merge those dictionaries (hint: https://stackoverflow.com/a/26853961) and then compare that to the response from bitcoin-cli -getinfo.

I think that we should be doing that instead of comparing against getinfo. This is explicitly a new thing which is similar to getinfo but not exactly replicating it.

@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 5, 2017

Member

I modified the test to compare against the get*info RPCs instead of getinfo

Member

achow101 commented Sep 5, 2017

I modified the test to compare against the get*info RPCs instead of getinfo

@jnewbery

one nit inline.

Please tidy up commits. First two commits should be squashed, and final two commits should be squashed into a second commit.

Show outdated Hide outdated src/rpc/protocol.cpp
@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 6, 2017

Member

Tidied up commits

Member

achow101 commented Sep 6, 2017

Tidied up commits

@jnewbery

Some comments in line.

You seem to have lost the test in your commit squashing.

Show outdated Hide outdated src/bitcoin-cli.cpp
Show outdated Hide outdated src/bitcoin-cli.cpp
Show outdated Hide outdated src/bitcoin-cli.cpp
@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 18, 2017

Member

Fixed the test failure.

Member

achow101 commented Sep 18, 2017

Fixed the test failure.

@jnewbery

Looks really good. Just a few nitty comments inline

Show outdated Hide outdated src/bitcoin-cli.cpp
Show outdated Hide outdated src/bitcoin-cli.cpp
Show outdated Hide outdated test/functional/bitcoin_cli.py
@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 26, 2017

Member

Addressed nits (sorry for the delay).

Member

achow101 commented Sep 26, 2017

Addressed nits (sorry for the delay).

@jnewbery

This comment has been minimized.

Show comment
Hide comment
@jnewbery

jnewbery Sep 26, 2017

Member

Tested ACK 5ca97e5.

Thanks for sticking with this!

Member

jnewbery commented Sep 26, 2017

Tested ACK 5ca97e5.

Thanks for sticking with this!

@TheBlueMatt

This comment has been minimized.

Show comment
Hide comment
@TheBlueMatt

TheBlueMatt Sep 28, 2017

Contributor
Contributor

TheBlueMatt commented Sep 28, 2017

laanwj and others added some commits Sep 29, 2016

rpc: Handle `getinfo` locally in bitcoin-cli w/ `-getinfo`
This adds the infrastructure `BaseRequestHandler` class that takes care
of converting bitcoin-cli arguments into a JSON-RPC request object, and
converting the reply into a JSON object that can be shown as result.

This is subsequently used to handle the `-getinfo` option, which sends
a JSON-RPC batch request to the RPC server with
`["getnetworkinfo", "getblockchaininfo", "getwalletinfo"]`,
and after reply combines the result into what looks like a `getinfo`
result.

There have been some requests for a client-side `getinfo` and this
is my PoC of how to do it. If this is considered a good idea
some of the logic could be moved up to rpcclient.cpp and
used in the GUI console as well.

Extra-Author: Andrew Chow <achow101@gmail.com>
Add test for bitcoin-cli -getinfo
Extra-Author: Andrew Chow <achow101@gmail.com>
@achow101

This comment has been minimized.

Show comment
Hide comment
@achow101

achow101 Sep 28, 2017

Member

Added the atomicity note to the help text.

Member

achow101 commented Sep 28, 2017

Added the atomicity note to the help text.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Sep 28, 2017

Member

utACK 5e69a43

Member

laanwj commented Sep 28, 2017

utACK 5e69a43

@laanwj laanwj merged commit 5e69a43 into bitcoin:master Sep 28, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Sep 28, 2017

Merge #10871: Handle getinfo in bitcoin-cli w/ -getinfo (revival of #…
…8843)

5e69a43 Add test for bitcoin-cli -getinfo (John Newbery)
3826253 rpc: Handle `getinfo` locally in bitcoin-cli w/ `-getinfo` (Wladimir J. van der Laan)

Pull request description:

  Since @laanwj doesn't want to maintain these changes anymore, I will.

  This PR is a revival of #8843. I have addressed @jnewbery's comments.

  Regarding atomicity, I don't think that is a concern here. This is explicitly a new API and those who use it will know that this is different and that it is not atomic.

Tree-SHA512: 9664ed13a5557bda8c43f34d6527669a641f260b7830e592409b28c845258fc7e0fdd85dd42bfa88c103fea3ecdfede5f81e3d91870e2accba81c6d6de6b21ff

@laanwj laanwj removed this from Blockers in High-priority for review Sep 28, 2017

@QingjingJing

This comment has been minimized.

Show comment
Hide comment
@QingjingJing

QingjingJing Sep 28, 2017

QingjingJing commented Sep 28, 2017

luke-jr added a commit to bitcoinknots/bitcoin that referenced this pull request Nov 6, 2017

rpc: Handle `getinfo` locally in bitcoin-cli w/ `-getinfo`
This adds the infrastructure `BaseRequestHandler` class that takes care
of converting bitcoin-cli arguments into a JSON-RPC request object, and
converting the reply into a JSON object that can be shown as result.

This is subsequently used to handle the `-getinfo` option, which sends
a JSON-RPC batch request to the RPC server with
`["getnetworkinfo", "getblockchaininfo", "getwalletinfo"]`,
and after reply combines the result into what looks like a `getinfo`
result.

There have been some requests for a client-side `getinfo` and this
is my PoC of how to do it. If this is considered a good idea
some of the logic could be moved up to rpcclient.cpp and
used in the GUI console as well.

Extra-Author: Andrew Chow <achow101@gmail.com>

Github-Pull: #10871
Rebased-From: 3826253
@crombiecrunch

This comment has been minimized.

Show comment
Hide comment
@crombiecrunch

crombiecrunch Feb 9, 2018

can someone please tell me how to apply this so that getinfo can work with the yiimp opensource pool files?

crombiecrunch commented Feb 9, 2018

can someone please tell me how to apply this so that getinfo can work with the yiimp opensource pool files?

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Feb 9, 2018

Member

@crombiecrunch Please submit a pull request upstream, replacing getinfo['balance'] with getwalletinfo['balance'] in the open source code.

Member

MarcoFalke commented Feb 9, 2018

@crombiecrunch Please submit a pull request upstream, replacing getinfo['balance'] with getwalletinfo['balance'] in the open source code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment