Break down CTest cases into each test suite#1931
Conversation
d2d0dcf to
12fd6a4
Compare
3e5ecac to
69ffaf4
Compare
|
This seems to save a couple of minutes on Windows, but we're still in the realm of 10 minutes for testing. Though at least CTest is nice enough to give us a (wall-clock) breakdown: ...which seems to imply that the slowest is running the external tests. For comparison, macOS (5m30s): |
69ffaf4 to
371e54d
Compare
|
Each external test is independent; could they at least be broken down into free and nonfree categories? |
371e54d to
eb36754
Compare
|
That sure would be nice! But the test runner currently doesn't allow splitting these two categories. And I've limited my effort in this PR to not touching the test runners at all, as a scope limiter. |
eb36754 to
9bfe422
Compare
| ASAN_OPTIONS=detect_leaks=0 ctest --test-dir build --schedule-random --label-exclude external | ||
| cmake --install build --verbose |
There was a problem hiding this comment.
If we're able to run ctest before cmake --install, then we don't need to be installing rgbds at all. That goes for the other OSes as well.
There was a problem hiding this comment.
I think installing is mostly used as a control for whether it works and in the correct locations. But, yeah, at this point we should probably just get rid of all --installs—in a follow-up PR, though.
9bfe422 to
fd9d366
Compare
This allows parallelising the individual programs' test suites, which can save a few minutes!
fd9d366 to
1491e05
Compare
Rangi42
left a comment
There was a problem hiding this comment.
There were some comments to follow up on in more PRs (getting rid of unnecessary install commands in CI, using ProcessorCount instead of hard-coded 4, consistent ctest flags instead of --label-exclude external-repos being unique) but this is good enough. Thank you!
This allows parallelising the individual programs' test suites, which seems to save around two minutes and a half!
This is a medium-effort preview of #1927: this doesn't expose each test to CTest, but at least we get some useful amount of parallelism out of this. (Especially for our CI.)