[tests] Introduce TestNode #10711

jnewbery · Jun 30, 2017

Continues #10082

TestNode is a class responsible for all state related to a bitcoind node
under test. It stores local state, is responsible for tracking the
bitcoind process and delegates unrecognised messages to the RPC
connection.

This commit changes start_nodes and stop_nodes to start and stop the
bitcoind nodes in parallel, making test setup and teardown much faster.

On my vm, this changeset reduces total test_runner runtime for the base set of tests
(including building the cache) from 263s to 195s (a 25% speedup). Note that the time
reported by test_runner does not include time spent building the cache:

with TestNode:

→ date +"%T" ; ./test_runner.py -q ; date +"%T"
12:48:04
..................................................................................................................................................................................................................................................................................................................................
TEST                           | STATUS    | DURATION

abandonconflict.py             | ✓ Passed  | 12 s
bip68-112-113-p2p.py           | ✓ Passed  | 19 s
blockchain.py                  | ✓ Passed  | 8 s 
bumpfee.py                     | ✓ Passed  | 13 s
decodescript.py                | ✓ Passed  | 3 s 
disablewallet.py               | ✓ Passed  | 3 s 
disconnect_ban.py              | ✓ Passed  | 6 s 
fundrawtransaction.py          | ✓ Passed  | 37 s
getchaintips.py                | ✓ Passed  | 4 s 
httpbasics.py                  | ✓ Passed  | 3 s 
import-rescan.py               | ✓ Passed  | 4 s 
importmulti.py                 | ✓ Passed  | 6 s 
importprunedfunds.py           | ✓ Passed  | 3 s 
invalidblockrequest.py         | ✓ Passed  | 4 s 
invalidtxrequest.py            | ✓ Passed  | 4 s 
keypool.py                     | ✓ Passed  | 7 s 
listsinceblock.py              | ✓ Passed  | 4 s 
listtransactions.py            | ✓ Passed  | 33 s
mempool_limit.py               | ✓ Passed  | 4 s 
mempool_persist.py             | ✓ Passed  | 15 s
mempool_reorg.py               | ✓ Passed  | 4 s 
mempool_resurrect_test.py      | ✓ Passed  | 3 s 
mempool_spendcoinbase.py       | ✓ Passed  | 3 s 
merkle_blocks.py               | ✓ Passed  | 3 s 
multi_rpc.py                   | ✓ Passed  | 4 s 
net.py                         | ✓ Passed  | 3 s 
nulldummy.py                   | ✓ Passed  | 3 s 
p2p-compactblocks.py           | ✓ Passed  | 28 s
p2p-fullblocktest.py           | ✓ Passed  | 126 s
p2p-leaktests.py               | ✓ Passed  | 8 s 
p2p-mempool.py                 | ✓ Passed  | 3 s 
p2p-segwit.py                  | ✓ Passed  | 59 s
p2p-versionbits-warning.py     | ✓ Passed  | 8 s 
preciousblock.py               | ✓ Passed  | 3 s 
prioritise_transaction.py      | ✓ Passed  | 5 s 
proxy_test.py                  | ✓ Passed  | 3 s 
rawtransactions.py             | ✓ Passed  | 9 s 
receivedby.py                  | ✓ Passed  | 19 s
reindex.py                     | ✓ Passed  | 12 s
rest.py                        | ✓ Passed  | 9 s 
rpcnamedargs.py                | ✓ Passed  | 3 s 
segwit.py                      | ✓ Passed  | 7 s 
sendheaders.py                 | ✓ Passed  | 24 s
signmessages.py                | ✓ Passed  | 3 s 
signrawtransactions.py         | ✓ Passed  | 3 s 
txn_clone.py                   | ✓ Passed  | 4 s 
txn_doublespend.py --mineblock | ✓ Passed  | 4 s 
uptime.py                      | ✓ Passed  | 3 s 
wallet-accounts.py             | ✓ Passed  | 3 s 
wallet-dump.py                 | ✓ Passed  | 7 s 
wallet-encryption.py           | ✓ Passed  | 8 s 
wallet-hd.py                   | ✓ Passed  | 15 s
wallet.py                      | ✓ Passed  | 31 s
walletbackup.py                | ✓ Passed  | 104 s
zapwallettxes.py               | ✓ Passed  | 9 s 
zmq_test.py                    | ○ Skipped | 0 s 

ALL                            | ✓ Passed  | 735 s (accumulated)
Runtime: 189 s

12:51:19

master:

→ date +"%T" ; ./test_runner.py -q ; date +"%T"
12:40:13
..........................................................................................................................................................................................................................................................................................................................................................................................................................................
TEST                           | STATUS    | DURATION

abandonconflict.py             | ✓ Passed  | 15 s
bip68-112-113-p2p.py           | ✓ Passed  | 19 s
blockchain.py                  | ✓ Passed  | 8 s
bumpfee.py                     | ✓ Passed  | 20 s
decodescript.py                | ✓ Passed  | 3 s
disablewallet.py               | ✓ Passed  | 3 s
disconnect_ban.py              | ✓ Passed  | 8 s
fundrawtransaction.py          | ✓ Passed  | 36 s
getchaintips.py                | ✓ Passed  | 11 s
httpbasics.py                  | ✓ Passed  | 7 s
import-rescan.py               | ✓ Passed  | 16 s
importmulti.py                 | ✓ Passed  | 10 s
importprunedfunds.py           | ✓ Passed  | 5 s
invalidblockrequest.py         | ✓ Passed  | 4 s
invalidtxrequest.py            | ✓ Passed  | 3 s
keypool.py                     | ✓ Passed  | 7 s
listsinceblock.py              | ✓ Passed  | 11 s
listtransactions.py            | ✓ Passed  | 37 s
mempool_limit.py               | ✓ Passed  | 4 s
mempool_persist.py             | ✓ Passed  | 23 s
mempool_reorg.py               | ✓ Passed  | 7 s
mempool_resurrect_test.py      | ✓ Passed  | 3 s
mempool_spendcoinbase.py       | ✓ Passed  | 3 s
merkle_blocks.py               | ✓ Passed  | 10 s
multi_rpc.py                   | ✓ Passed  | 6 s
net.py                         | ✓ Passed  | 6 s
nulldummy.py                   | ✓ Passed  | 3 s
p2p-compactblocks.py           | ✓ Passed  | 30 s
p2p-fullblocktest.py           | ✓ Passed  | 126 s
p2p-leaktests.py               | ✓ Passed  | 8 s
p2p-mempool.py                 | ✓ Passed  | 3 s
p2p-segwit.py                  | ✓ Passed  | 62 s
p2p-versionbits-warning.py     | ✓ Passed  | 8 s
preciousblock.py               | ✓ Passed  | 8 s
prioritise_transaction.py      | ✓ Passed  | 7 s
proxy_test.py                  | ✓ Passed  | 10 s
rawtransactions.py             | ✓ Passed  | 15 s
receivedby.py                  | ✓ Passed  | 28 s
reindex.py                     | ✓ Passed  | 12 s
rest.py                        | ✓ Passed  | 12 s
rpcnamedargs.py                | ✓ Passed  | 3 s
segwit.py                      | ✓ Passed  | 12 s
sendheaders.py                 | ✓ Passed  | 26 s
signmessages.py                | ✓ Passed  | 3 s
signrawtransactions.py         | ✓ Passed  | 3 s
txn_clone.py                   | ✓ Passed  | 10 s
txn_doublespend.py --mineblock | ✓ Passed  | 10 s
uptime.py                      | ✓ Passed  | 3 s
wallet-accounts.py             | ✓ Passed  | 3 s
wallet-dump.py                 | ✓ Passed  | 6 s
wallet-encryption.py           | ✓ Passed  | 8 s
wallet-hd.py                   | ✓ Passed  | 18 s
wallet.py                      | ✓ Passed  | 69 s
walletbackup.py                | ✓ Passed  | 130 s
zapwallettxes.py               | ✓ Passed  | 15 s
zmq_test.py                    | ○ Skipped | 0 s

ALL                            | ✓ Passed  | 936 s (accumulated)
Runtime: 242 s

12:44:36

jnewbery · Jun 30, 2017

Wallet tests were failing on Travis because bitcoind startup was timing out. Increasing timeout from 2.5s to 10s for Travis.

MeshCollider · Jul 4, 2017

Concept ACK e538dbf, looks good but haven't tested or gone through thoroughly

laanwj · Jul 10, 2017

The speed-up is nice, also like the idea of launching nodes in parallel. Concept ACK.

jtimon · Jul 10, 2017

jtimon reviewed Jul 10, 2017

View changes

Remove this?

thanks. Removed

jnewbery · Jul 11, 2017

fixed @jtimon's review comment and rebased.

jnewbery · Jul 24, 2017

rebased with TestNode changes for multiwallet.

jnewbery referenced this pull request Jun 30, 2017
Open
[tests] Demostration/concept: Add a TestNode class to test_Framework.py #10082
3 of 6 tasks complete

fanquake added the Tests label Jun 30, 2017

jnewbery referenced this pull request Jul 11, 2017
Open
[tests] [utils] test bitcoin-cli #10798

laanwj removed from Blockers in High-priority for review Jul 13, 2017

@@ -244,6 +244,7 @@ def get_auth_cookie(datadir, n):
                      if line.startswith("rpcpassword="):
                          assert password is None  # Ensure that there is only one rpcpassword line
                          password = line.split("=")[1].strip("\n")
+    +    # print(os.path.join(datadir, "regtest", ".cookie"))

bitcoin/bitcoin

[tests] Introduce TestNode #10711

jnewbery commented Jun 30, 2017

jnewbery referenced this pull request Jun 30, 2017

[tests] Demostration/concept: Add a TestNode class to test_Framework.py #10082

fanquake added the Tests label Jun 30, 2017

jnewbery commented Jun 30, 2017

MeshCollider commented Jul 4, 2017

laanwj commented Jul 10, 2017 • edited

jtimon reviewed Jul 10, 2017

View changes

jnewbery commented Jul 11, 2017

jnewbery referenced this pull request Jul 11, 2017

[tests] [utils] test bitcoin-cli #10798

laanwj removed from Blockers in High-priority for review Jul 13, 2017

jnewbery added some commits Jun 2, 2017

jnewbery commented Jul 24, 2017

bitcoin/bitcoin

Join GitHub today

[tests] Introduce TestNode #10711

Conversation

jnewbery commented Jun 30, 2017

jnewbery referenced this pull request Jun 30, 2017

[tests] Demostration/concept: Add a TestNode class to test_Framework.py #10082

fanquake added the Tests label Jun 30, 2017

jnewbery commented Jun 30, 2017

MeshCollider commented Jul 4, 2017

laanwj commented Jul 10, 2017 • edited

jtimon reviewed Jul 10, 2017 View changes

jtimon Jul 10, 2017

jnewbery Jul 11, 2017

jnewbery commented Jul 11, 2017

jnewbery referenced this pull request Jul 11, 2017

[tests] [utils] test bitcoin-cli #10798

laanwj removed from Blockers in High-priority for review Jul 13, 2017

jnewbery added some commits Jun 2, 2017

jnewbery commented Jul 24, 2017

jtimon reviewed Jul 10, 2017

View changes