Testing

###What's needed before starting:

An installation of Bitcoin Core (it doesn't have to be running or synchronized, as we only use regtest mode here).

You'll need to know the path to your bitcoind and bitcoin-cli commands (see below), although they will be picked up as default if you miss out the btcroot option.

Note: Commands prefaced with $ are run from the root joinmarket directory.

Create a default joinmarket config:
$ cp test/regtest_joinmarket.cfg joinmarket.cfg

Create a testing-specific bitcoin_test.conf: Linux:
$ touch ~/.bitcoin/bitcoin.conf ~/.bitcoin/bitcoin_test.conf

OSX:
$ touch $HOME/Library/Application\ Support/Bitcoin/bitcoin_test.conf

Edit the above file to match your joinmarket.conf rpcuser and rpcpassword:

#rpc_user value from joinmarket.conf
rpcuser=bitcoinrpc
#rpc_password value from joinmarket.conf
rpcpassword=123456abcdef
#testing-specific port
rpcport=62612
walletnotify=curl -sI --connect-timeout 0.1 http://localhost:62612/walletnotify?%s
alertnotify=/usr/bin/wget -q --spider --timeout=0.5 --tries=1 http://localhost:62612/alertnotify?%s

Install miniircd (lightweight IRC server):
$ git clone https://github.com/Joinmarket-Org/miniircd

###Running tests

The command given in the main README runs a full test suite (not including tumbler - if you remove the "ignore" option it will include this, and take an extra 5-10 minutes):

PYTHONPATH=.:$PYTHONPATH py.test --cov-report html --btcroot=/path/to/bitcoin/bin/ --btcconf=/path/to/bitcoin.conf --btcpwd=123456abcdef --nirc=2 --ignore test/test_tumbler.py

This (or similar) will show the coverage after the run:

open htmlcov/index.html

Since a full test suite takes several minutes, you may want to just run individual tests (such as one you write); in that case, just do:

PYTHONPATH=.:$PYTHONPATH py.test --cov-report html --btcroot=/path/to/bitcoin/bin/ --btcconf=/path/to/bitcoin.conf --btcpwd=123456abcdef --nirc=2 test/test_mytest.py

Note the pytest convention: test modules should start with test_* and test functions in test modules, likewise.

The tests are based on two key tools: miniircd and Bitcoin regtest. The former is an ultra-lightweight IRC server which is run locally. The latter is Bitcoin's feature to allow testing end-to-end bitcoin transactions and blockchain access again, locally on your own machine, with control over block mining. The current list of tests covers:

• test_enc_wrapper.py - the libnacl e2e encryption for messages

• test_bip32.py - the BIP 32 HD wallet key manipulation library

• test_addresses.py - testing Bitcoin address handling

• test_regtest.py - an end to end test of sendpayment with yield generators running in the background

• test_mnemonic.py - testing the old electrum 12 word mnemonic seed generation used in joinmarket

• test_blockr.py - tests wallet access via the blockr blockchain interface

• test_wallets.py - testing wallet passwords and recovery

• test_btc_formatting.py - some tests of data parsing for Bitcoin

• test_donations.py - testing the special reusable pubkey ("stealth") style payments for tumbler donations

• test_ecc_signing.py - test ECDSA signatures with RFC6979 against test vectors

• test_tumbler.py - test of the tumbler bot

• test_tx_serialize.py - testing bitcoin transaction serialization and deserialization

• test_tx_creation.py - test correct complete construction of various Bitcoin transactions

• test_keys.py - tests handling of WIF private keys

• test_broadcast_method.py - tests broadcast via self. vs other bots

• test_podle.py - tests creation of PoDLE style commitments used as of 0.2.0

• conftest.py - special pytest file used for test setup (in particular, startup/shutdown of bitcoin regtest blockchain and miniircd IRC server)

A number of test vectors are stored in *.json files in the test/ directory also.

###Some features of pytest used:

The build test in travis.yml is basically the command listed above, so according to pytest rules it picks up all the test_* modules in the test directory.

pytest fixtures can be used for setup/teardown of individual test modules, as opposed to the global setup in conftest.py. Mainly this is calling things like load_program_config.

Another very useful feature is test parametrization. For example, from test_regtest.py:

@pytest.mark.parametrize(
"num_ygs, wallet_structures, mean_amt, mixdepth, sending_amt",
[
    # basic 1sp 2yg
    (2, [[1, 0, 0, 0, 0]] * 3, 10, 0, 100000000),
    # 1sp 4yg, 2 mixdepths
    (4, [[1, 2, 0, 0, 0]] * 5, 4, 1, 1234500),
    # 1sp 3yg, 2 mixdepths, sweep from depth1
    (3, [[1, 3, 0, 0, 0]] * 4, 4, 1, 0),
])

Each tuple in the list is the set of arguments to re-run the test function against. So here we are trying the sendpayment in a few different modes. We can add tons more, but have to consider how long it takes.

###Test scripts not included in the build test:

test_tumbler.py - even the simplest semi-realistic tumbler test takes 5 minutes+ to complete so is probably not practical as part of a build test, hence not added to the travis build.

###Tests being worked on or still needed:

One area badly needing more tests is the BlockchainInterface implementations. These are especially tricky as they can't be done against a Bitcoin regtest instance. There is currently test_blockr.py which is covering some features against the testnet version. Finding more sophisticated tests that don't require "real" blockchain access is an open question.

We need more thorough testing of IRC functioning (see test_irc_messaging.py), although a lot has been done.

Related (and more important architecturally), we need more testing of the Joinmarket messaging protocol (e.g. feeding garbage messages to bots). There is some of that in the above mentioned test script.

###Clean up.

If you are doing a lot of tests, your regtest blockchain gets a bit crowded; generally this needn't be a problem, although I have occasionally seen delays and glitches. At least periodically, you'll want to wipe your regtest blockchain:

rm -rf /home/joinmarketeer/.bitcoin/regtest

Needless to say, be very careful doing this.

###Some useful/not obvious things for test writers

The file test/commontest.py contains helper functions, at the moment the only important one is make_wallets, which is a convenient way to populate a set of wallets in regtest mode with coins at various mixdepths. See the documentation of the function for details.

A very unobvious point, related to pytest: pytest does provide coverage of subprocesses. This is very useful in our case, because bots like sendpayment and yieldgenerator work on listening loops, so cannot run in the same path of execution. We start up one bot in the main process and other bots with Python's subprocess module; but, when shutdown is needed, it must be done "gracefully", otherwise pytest's coverage post-processing won't be able to operate. See these lines in test_regtest.py:

finally:
    if any(yigen_procs):
        for ygp in yigen_procs:
            #NB *GENTLE* shutdown is essential for
            #test coverage reporting!
            ygp.send_signal(signal.SIGINT)
            ygp.wait()