[tests] [utils] test bitcoin-cli #10798

Open
wants to merge 3 commits into
from

Conversation

Projects
None yet
2 participants
Member

jnewbery commented Jul 11, 2017 edited

We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced.

Let's fix that.

This PR adds bitcoin-cli testing in the python functional test_framework:

  1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of getinfo is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features

EDIT: --usecli option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.

  1. Add a --usecli option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:
  • the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC
  • we're sending a very large RPC request (eg submitblock) and it can't be serialized into a shell bitcoin-cli call.

I think that even though --usecli doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:

  • enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the --usecli option is used.
  • run a subset of tests in Travis with -usecli

This builds on and requires the TestNode PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class.

Addresses #10791

jnewbery changed the title from test bitcoin-cli to [tests] [utils] test bitcoin-cli Jul 11, 2017

fanquake added the Tests label Jul 11, 2017

jnewbery added some commits Jun 2, 2017

@jnewbery jnewbery [tests] Introduce TestNode
TestNode is a class responsible for all state related to a bitcoind node
under test. It stores local state, is responsible for tracking the
bitcoind process and delegates unrecognised messages to the RPC
connection.

This commit changes start_nodes and stop_nodes to start and stop the
bitcoind nodes in parallel, making test setup and teardown much faster.
fdf769b
@jnewbery jnewbery [tests] add TestNodeCLI class for calling bitcoin-cli for a node 8cdcf5e
@jnewbery jnewbery [tests] Add bitcoin_cli.py test script 7e12886
Member

jnewbery commented Jul 13, 2017

Changed subprocess.run() to subprocess.check_output() to be compatible with pre3.5 python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment