Some input regarding generation node, regtest, ...

[dexX7]

Just yesterday I stumbled upon your talk about unit and regtests tests and I was quite impressed.

I noticed a few points you mentioned:

• the use of shell scripts
• preserving an initial state, so it's not required to regenerate the first 100+ blocks on every run
• the initial Bitcoins are associated with "accounts"

Very recently, I think, Bitcoin Core master basically build a Python framework for regtests that takes care of a few things in particular:

• nodes are spawned via Python and they are completely isolated
• on the first startup 200 blocks are generated and the results are stored in ./cache
• those results are then copied and reused to working dirs /tmp/testXXX for each node
• after a test run, the temporary files /tmp/testXXX are deleted

My initial goal was to port the few regtests of Master Core v0.0.8, but I ended up modifiying the framework and basically hacked a Master Core regtest framework together. In the context of above:

• I use one single node to generate blocks and properties. If another node needs some BTC, then this node is used as provider. I decided this is cleaner than letting all four nodes (as per default with the core framework) generating blocks.
• I added a method that collects all unspent outputs and then sends them to an address to get full control and get rid of those Bitcoin "accounts".

I haven't explored bitcoin-spock yet (unfortunately!), and I'm not sure, if this is of any help, but wanted to let you know as soon as possible. :)

Regarding consensus hashes, I started mastercoin-MSC/mastercore#143 some time ago.

[msgilligan]

Thanks, @dexX7. I'm hoping to find some time to return to working on these tests soon. I'd also like to look into running your tests from the Mastercoin Jenkins server.

[dexX7]

Since there are a few points where I'm not fully satisfied with my current solution (and likewise the whole Bitcoin Core Python testing framework), I'd really like to look into bitcoin-spock rather sooner than later, especially due to the very descriptive nature of the tests I've seen here.

Excuse my probably obvious questions and please correct me, if I'm wrong: a) tests can be run locally, b) Jenkins is not required to run tests and c) Jenkins is manager for automated test, similar to Travis CI, correct?

The current process is as follows:

1. Initial preparation: a "cache", with subfolders for each node, is created.
2. Four nodes spawn, with datadirs pointed to the cache location.
3. Node 1 generates 200 blocks.
4. All nodes shut down and the initial setup ends. Basically what would end up in ~/.bitcoin/regtest as per default, can now be found in ./cache/node1/regtest, ./cache/node2/regtest, ... This is done once, or after an explicit cleanup, and the content of "cache" is going to be reused.
5. Test preparation: a temporary folder is created (/tmp/testX where X is random, as per default) and the content of "cache" is copied.
6. Network setup: Four nodes spawn, with datadirs pointed to the temporary location.
7. All nodes connect to each other.
8. Test: commands are issued via the RPC interface of each node. Results are checked manually and an exception is thrown, if something is wrong.
9. Happy route: if there was no exception, all nodes shut down.
10. Otherwise, and if not caught within the test, any exception is caught, but the testing ends here nevertheless.
11. Cleanup: temporary files and folders are deleted.
12. Termination: the program ends and returns an exit code (note to myself: actually do return an exit code!).

I assume an integration would start by pulling the tests and executing each of them. Would it be more favorable to have one test that calls sub tests or should this be done one by one, repeating the process starting at step 5? What can be done for a frictionless integration and what might be improved, assuming it's not droped in favor of an alternative?

Ah, and as a side note: I also started to create unit tests for Master Core, based on the existing Boost tests. One of the tests fails, but the tested behavior isn't used, nor necessary, to run Master Core successfully. Does this have a negative impact on the Jenkins testing and would it make sense to remove this specific test?

[msgilligan]

@dexX7 : You are correct on all counts:
a) Spock tests can be run locally. It's pretty easy. See the section Running the Tests Manually in the README. The only prerequisite is Java 7 or later.
b) No Jenkins required -- it does seem that some folks at MSC are confused about that, though.
c) Yes, Jenkins is a server process that monitors Git, runs test jobs, and collects results. For Mastercoin, it's running on ci.mastercoin.info

The tests in bitcoin-spock currently only expect a single bitcoind/mastercored node running either on MainNet or in RegTest mode. They could (and should) be extended to run on TestNet and to test block chain forks with RegTest and multiple nodes.

It is definitely possible to use Jenkins to run your python RegTest process using the Python tests.

I think I'd have to look at your python tests to determine what would be required for an integration and I'm really busy for the next week or two. I'm really hoping got get back to the Spock tests soon and that would be a good time to look at your tests. I also will make time to answer any questions you have about the Spock tests. You should definitely try to run them locally.

For the failing Boost test, it should probably be temporarily disabled (either in the code or with a custom Jenkins configuration) so that the Jenkins Master Core builds aren't marked as bad.

[dexX7]

Thanks! This is very helpful.

[msgilligan]

I think it's time to close this issue. @dexX7 is there anything in here that warrants leaving this open or creating new issues?

[dexX7]

Yes, can be closed.