Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
PartialTestResult.join_results(result, pickle.load(input)) EOFError libgomp: Thread creation failed: Resource temporarily unavailable #2807
I'm experimenting with parallel execution of tests in the Fedora buildsystem (building Cython 0.29.3).
Up until now the test were disabled because of #1982 but I have decided to enable them and only skip the failing tests on Big Endian. The test are quite slow so I've decided to use
The number of CPUs is however quite arbitrary and differs with each build. A i686 builder that was picked god 48 CPUs, so it used
We run python2 tests before python3 tests, so this is where I got a strange error. Let me know if I shall reverse the order to see if this happens on Python 3 as well.
Full log: build.log
This error did not occur another time when the builder had just 6 CPUs and
I've tried to limit the number to 16, however I got the same error with
Currently I'm experimenting with
I've only experienced this on i686, yet this was the only builder that I got with 48 CPUs this time. A x86_64 build with
My very wild guess is that with massive parallelism, the IO is not so fast and something reads a pickle jar too soon.
OK, this is getting weird. On this machine:
It works even with
But on this one:
It fails even with
Let me check if it doesn't fail with
added a commit
Jan 19, 2019
According to the log (thanks for providing the full output), the test is failing with this error:
That suggests that OpenMP fails to start its threads for some reason. I would recommend reducing the number of processes relative to the number of cores, since the test runner will also fork out the test runs and some tests will start threads or further subprocesses.
This page also suggests that passing
changed the title
runtests.py -j48: PartialTestResult.join_results(result, pickle.load(input)) EOFError
Jan 19, 2019
The log currently ends with: