Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run the tests in parallel #3777

Merged
merged 3 commits into from Jul 2, 2020
Merged

Run the tests in parallel #3777

merged 3 commits into from Jul 2, 2020

Conversation

@regnat
Copy link
Contributor

regnat commented Jul 2, 2020

Change the way the testsuite runs to make each test a separate make target, allowing to run them in parallel rather than sequentially as we do now.
This causes make installcheck -j$(nproc) to drop from ~4m30 to ~40s on my machine.

To allow that, each test now runs under a different TEST_ROOT (which also ensures that they are truly independant), and runs tests/init.sh first to setup everything

regnat added 3 commits Jul 2, 2020
Cause the time needed to run the testsuite to drop from ~4mins to ~40s
Use a fifo pipe to handle the synchronisation between the different
threads rather than relying on delays
Use a fifo pipe to handle the synchronisation between the different
threads rather than relying on delays
@edolstra edolstra merged commit ec5d7cb into NixOS:master Jul 2, 2020
0 of 2 checks passed
0 of 2 checks passed
tests (ubuntu-latest) tests (ubuntu-latest)
Details
tests (macos-latest) tests (macos-latest)
Details
@edolstra edolstra deleted the tweag:parallel-tests branch Jul 2, 2020
@edolstra
Copy link
Member

edolstra commented Jul 2, 2020

Thanks, awesome :-)

@regnat
Copy link
Contributor Author

regnat commented Jul 2, 2020

Bleh, this broke OSX tests because the socket path becomes too long on the CI 😒 I'll open a MR to increase its length

@edolstra
Copy link
Member

edolstra commented Jul 2, 2020

I got this running nix-build release.nix -A build.x86_64-linux:

ran test tests/plugins.sh... [FAIL]
    + set -o pipefail
    ++ nix eval '(builtins.anotherNull)' --option setting-set true --option plugin-files '/build/source/tests/plugins/libplugintest*'
    error: --- SysError --- nix
    opening directory '/build/source/tests/plugins/libplugintest*': No such file or directory
    + res=
make: *** [mk/lib.mk:111: tests/plugins.sh.test] Error 1
make: *** Waiting for unfinished jobs....
ran test tests/brotli.sh... [PASS]
@regnat
Copy link
Contributor Author

regnat commented Jul 2, 2020

Mh nvm, I can't do that, the size is defined in glibc 🤦

I'll just try and shorten a bit the TEST_ROOT

@regnat
Copy link
Contributor Author

regnat commented Jul 2, 2020

I got this running nix-build release.nix -A build.x86_64-linux:

ran test tests/plugins.sh... [FAIL]
    + set -o pipefail
    ++ nix eval '(builtins.anotherNull)' --option setting-set true --option plugin-files '/build/source/tests/plugins/libplugintest*'
    error: --- SysError --- nix
    opening directory '/build/source/tests/plugins/libplugintest*': No such file or directory
    + res=
make: *** [mk/lib.mk:111: tests/plugins.sh.test] Error 1
make: *** Waiting for unfinished jobs....
ran test tests/brotli.sh... [PASS]

Mh I was indeed missing a dependency.

regnat added a commit to tweag/nix that referenced this pull request Jul 2, 2020
Reuse the pre-existing list rather than the one written as part of NixOS#3777
@regnat regnat mentioned this pull request Jul 2, 2020
@regnat
Copy link
Contributor Author

regnat commented Jul 2, 2020

#3778 fixes the build.x86_64-linux. I'm not sure how to reliably solve the OSX CI error as it looks like we had a very small margin wrt the maximum socket path length

@Ericson2314
Copy link
Member

Ericson2314 commented Jul 3, 2020

YAY!!!!! Wow, I hindsight I have no idea how I survived the past month without this :) Thanks @regnat!!!

regnat added a commit to tweag/nix that referenced this pull request Jul 3, 2020
Reuse the pre-existing list rather than the one written as part of NixOS#3777
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.