New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_selectors.PollSelectorTestCase.test_above_fd_setsize reported killed by shell #66100
Comments
On one particular linux vserver virtual machine (which is unfortunately my development platform for python), test.test_selectors.PollSelectorTestCase.test_above_fd_setsize fails with the following message: zsh: killed and at that point the test suite stops running, regardless of whether or not I started it with -j. As far as I can tell, the configuration of this vserver is the same as the one my buildbots run on, but they are on different host machines, so there could be some differences I'm not remembering. On the buldbots, the test gets skipped with the message 'FD limit reached'. Anyone have any clues how to debug this? |
The test changes the maximum number of open files. What is the limit in your shell? You can try to modify the test to add print(soft, hard) after getrlimit(). On Fedora 20: $ python -c 'import resource; print(resource.getrlimit(resource.RLIMIT_NOFILE))'
(1024, 4096) The test tries to use the hard limit (4096) to set the soft limit (1024). |
There's probably a special mechanism due to vserver which makes the What happens if you try the following: You could run the process under strace to see what's going on: you'll |
ping? |
The python command just returns. The dmesg was a good call: python invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0 I *thought* I had this virtual server configured with the same resources as I do the buildbots, but I could be wrong. It's been quite some time since I set both of them up, and I don't even remember how the resources are set at the moment. Let me know if you want to see the entire dmesg output. |
340 MB to run test_selectors sounds high. What is the value of NUM_FDS? And what is the result of this command in your vserver? $ python -c 'import resource; print(resource.getrlimit(resource.RLIMIT_NOFILE))'
(1024, 4096) |
rdmurray@pydev:~/python/p34>python -c 'import resource; print(resource.getrlimit(resource.RLIMIT_NOFILE))' Unfortunately the buildbot box is offline at the moment and it may be a bit before I can get it back, so I can't compare the results above with that VM. |
Oh, 1 million files is much bigger than 4 thousand files (4096). The test should only test FD_SETSIZE + 10 files, the problem is to get FD_SETSITE:
For example, on my Linux FD_SETSIZE is 1024, whereas the hard limit of RLIMIT_NOFILE is 4096. /usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024 Maybe we can simply expose the FD_SETSIZE constant in the select module? The constant is useful when you use select.select(), which is still heavily used on Windows. |
We could cap it to let's say 2**16, it's larger than any possible But I don't see anything wrong with the test, it's really the buildbot |
That is the only test that fails for lack of memory. And it's not the buildbot, it's my development virtual machine. Having the test suite be killed when I do a full test run is...rather annoying. |
Alright, I'll cap the value then (no need to expose FD_SETSIZE). |
New changeset 7238c6a05ca6 by Charles-François Natali in branch '3.4': New changeset 89665cc05592 by Charles-François Natali in branch 'default': |
Sorry for the delay, should be fixed now. |
Test passes for me now, thanks. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: