Tests are the lifeblood of modern software engineering. Whenever you add or remove a feature you must ensure that the tests pass. Sadly, writing tests is as difficult as writing software, and tests are prone to misbehaviour. What follows is a short guide on how to write tests, how to update the tests, and basic troubleshooting.
A test is considered flaky if its pass/fail is not deterministic. In other words, there is no guarantee that it will always pass or always fail when you run it.
This file should be updated after any performance critical
changes. Note, because most flaky tests are of the integration
variety, updating the flakyness report can take upwards of 10 hours
given the minimum of 300 repetitions.
Unfortunately it is impossible to expedite the process, and doing something in parallel to the testing can worsen the results, so beware of repeating the tests on your work machine.
| Commit_sha | unstable…7_p…_1… | Two_networks | network_stable_after_add… | integration::p2p::multiple_networks | time_trigger_execution_count…10_percent | client_has_rejected_and_acepted_txs_should_return_tx_history |
|---|---|---|---|---|---|---|
| 938a3168387bb5a67aac19057d94c220c137dd96 | 1/300 | 3/300 | 5/300 | 0/300 | 0/300 | 0/300 |
| 3f88232ce36e441c7decdec86feae027d271e3ef | 2/300 | 5/300 | 6/300 | 1/300 | 1/300 | 1/300 |
| ba4571d96bea31cd9dea93128bf122e06cb43e31 | 1/300 | 4/300 | 2/300 | 1/300 | 0/300 | 0/300 |
| cf212d47e9426a1d7c4ed7a3aa8d16e885507f40 | 0/300 | 6/300 | 0/300 | 2/300 | 0/300 | 0/300 |