Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving testing speed #219

Closed
johnsonjh opened this issue Nov 18, 2020 · 4 comments
Closed

Improving testing speed #219

johnsonjh opened this issue Nov 18, 2020 · 4 comments

Comments

@johnsonjh
Copy link

While the tests complete in here in less than a minute, @cjdelisle advised they are taking close to 90s for him - on Travis, they can sometimes take 10 minutes or more, depending on the load.

Here's the runtime breakdown:

testtimes

The Neutrino test is 16s because it reliant on wall-clock time to do it's ban/unban testing, and is the low-hanging fruit. This test could be rewritten based on a range, or moved completely out into ALL_TESTS=1 (Neutrino has some slow, extra tests there already, including the tests that would require getting the full test harness working again.)

Anyway, for my 50s test runtime, the top 10 here are 45s of it.

@johnsonjh
Copy link
Author

Also, the chain database and wallet db tests can likely be "optimized" by disabling fsync.

@johnsonjh
Copy link
Author

These results are based on my fork, which has some PR's applied that aren't yet in develop, so my addrmanager/connmanager test is likely faster here and will get a bump once I rebase #212

@johnsonjh
Copy link
Author

@cjdelisle Would you be against adding a SLOW_TESTS variable and moving the Neutrino BanMan and SeedWords tests to only be active when it's defined, line LDB_TEST?

That should take 20s off right away.

@johnsonjh
Copy link
Author

I've fully implemented this - see #229 (comment) - it works very well without much loss of coverage. The link has all the details.

Also, I've found it's not that difficult to provide fake time data during tests so I'll soon be fixing the Neutrino BanMan test. Instead of taking 16s (or even 5s) it should be able to complete in 0.1-0.2s and won't ever throw any false positives in a CI/CD pipeline again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant