Add resource.prlimit #60799

tiran · 2012-12-02T11:14:42Z

BPO	16595
Nosy	@pitrou, @vstinner, @giampaolo, @tiran, @vajrasky
Files	prlimit2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2013-12-10.18:40:35.123>
created_at = <Date 2012-12-02.11:14:41.848>
labels = ['extension-modules', 'type-feature']
title = 'Add resource.prlimit'
updated_at = <Date 2013-12-10.18:40:35.121>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2013-12-10.18:40:35.121>
actor = 'pitrou'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2013-12-10.18:40:35.123>
closer = 'pitrou'
components = ['Extension Modules']
creation = <Date 2012-12-02.11:14:41.848>
creator = 'christian.heimes'
dependencies = []
files = ['30685']
hgrepos = []
issue_num = 16595
keywords = ['patch']
message_count = 20.0
messages = ['176783', '176785', '191759', '192578', '193361', '193367', '200737', '200901', '200902', '200906', '200910', '201203', '201206', '201207', '201208', '201236', '203505', '205841', '205842', '205843']
nosy_count = 8.0
nosy_names = ['pitrou', 'vstinner', 'giampaolo.rodola', 'christian.heimes', 'Arfrever', 'neologix', 'python-dev', 'vajrasky']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue16595'
versions = ['Python 3.4']

tiran · 2012-12-02T11:14:40Z

Linux has an additional function to setrlimit and getrlimit which supports to query and/or modify resource limits of another process.

http://www.kernel.org/doc/man-pages/online/pages/man2/getrlimit.2.html

The patch implements resource.prlimit(pid, resource[, limits]) on all platforms that have a prlimit() function. I haven't included the regenerated configure script. You have to run autoreconf in order to test the new function.

tiran · 2012-12-02T11:34:29Z

prlimit() needs glibc 2.13+ (thanks Arfrever).

tiran · 2013-06-24T13:11:59Z

Updated patch, now raises PermissionError on EPERM.

tiran · 2013-07-07T18:33:25Z

Does anybody want to review the code before I commit it?

giampaolo · 2013-07-19T14:46:16Z

From my original post which unfortunately was ignored :( :
http://mail.python.org/pipermail/python-ideas/2012-June/015323.html
...for completeness perhaps it also makes sense to expose Linux-specific RLIMIT constants:

RLIMIT_MSGQUEUE
RLIMIT_NICE
RLIMIT_RTPRIO
RLIMIT_RTTIME
RLIMIT_SIGPENDING

vajrasky · 2013-07-19T16:31:00Z

Fedora 18, this is the result after applying your patch and execute your unit test:

[sky@localhost cpython]$ ./python Lib/test/test_resource.py
test_args (main.ResourceTest) ... ok
test_fsize_enforced (main.ResourceTest) ... ok
test_fsize_ismax (main.ResourceTest) ... ok
test_fsize_toobig (main.ResourceTest) ... ok
test_getrusage (main.ResourceTest) ... ok
test_prlimit (main.ResourceTest) ... ERROR
test_setrusage_refcount (main.ResourceTest) ... ok

======================================================================
ERROR: test_prlimit (main.ResourceTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_resource.py", line 131, in test_prlimit
    1, resource.RLIMIT_AS)
  File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 614, in assertRaises
    return context.handle('assertRaises', callableObj, args, kwargs)
  File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 150, in handle
    callable_obj(*args, **kwargs)
PermissionError: [Errno 1] Operation not permitted

Ran 7 tests in 0.990s

FAILED (errors=1)
Traceback (most recent call last):
  File "Lib/test/test_resource.py", line 143, in <module>
    test_main()
  File "Lib/test/test_resource.py", line 140, in test_main
    support.run_unittest(ResourceTest)
  File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1615, in run_unittest
    _run_suite(suite)
  File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1590, in _run_suite
    raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File "Lib/test/test_resource.py", line 131, in test_prlimit
    1, resource.RLIMIT_AS)
  File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 614, in assertRaises
    return context.handle('assertRaises', callableObj, args, kwargs)
  File "/home/sky/Code/python/programming_language/cpython/Lib/unittest/case.py", line 150, in handle
    callable_obj(*args, **kwargs)
PermissionError: [Errno 1] Operation not permitted

With sudo, I got this:

[sky@localhost cpython]$ sudo ./python Lib/test/test_resource.py
[sudo] password for sky:
test_args (main.ResourceTest) ... ok
test_fsize_enforced (main.ResourceTest) ... ok
test_fsize_ismax (main.ResourceTest) ... ok
test_fsize_toobig (main.ResourceTest) ... ok
test_getrusage (main.ResourceTest) ... ok
test_prlimit (main.ResourceTest) ... FAIL
test_setrusage_refcount (main.ResourceTest) ... ok

======================================================================
FAIL: test_prlimit (main.ResourceTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib/test/test_resource.py", line 131, in test_prlimit
    1, resource.RLIMIT_AS)
AssertionError: ValueError not raised by prlimit

Ran 7 tests in 1.002s

FAILED (failures=1)
Traceback (most recent call last):
  File "Lib/test/test_resource.py", line 143, in <module>
    test_main()
  File "Lib/test/test_resource.py", line 140, in test_main
    support.run_unittest(ResourceTest)
  File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1615, in run_unittest
    _run_suite(suite)
  File "/home/sky/Code/python/programming_language/cpython/Lib/test/support.py", line 1590, in _run_suite
    raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File "Lib/test/test_resource.py", line 131, in test_prlimit
    1, resource.RLIMIT_AS)
AssertionError: ValueError not raised by prlimit

tiran · 2013-10-21T09:42:26Z

Thanks for your tests. I may have to rework my test scenario a bit.

python-dev · 2013-10-22T09:22:31Z

New changeset 796c21e27a92 by Christian Heimes in branch 'default':
Issue bpo-16595: Add prlimit() to resource module
http://hg.python.org/cpython/rev/796c21e27a92

tiran · 2013-10-22T09:27:12Z

Let's see how the buildbots respond. I may have to disable some tests for ancient Kernel versions.

pitrou · 2013-10-22T09:39:20Z

+ self.assertRaises(PermissionError, resource.prlimit,
+ 1, resource.RLIMIT_AS)

Please skip this when run as root.

+ self.assertEqual(resource.prlimit(0, resource.RLIMIT_AS), (-1, -1))

What if it's not that value by default? Please fix the test to make it
more robust.

tiran · 2013-10-22T09:55:26Z

Roundup bot hangs. I have addressed your concerns in http://hg.python.org/cpython/rev/8c77117f41a9

neologix · 2013-10-25T04:52:08Z

The test is failing with ENOSYS on one of the buildbots:
"""
======================================================================
ERROR: test_prlimit (test.test_resource.ResourceTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/test/test_resource.py", line 148, in test_prlimit
    1, resource.RLIMIT_AS)
  File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/unittest/case.py", line 689, in assertRaises
    return context.handle('assertRaises', callableObj, args, kwargs)
  File "/var/lib/buildslave/3.x.murray-gentoo/build/Lib/unittest/case.py", line 158, in handle
    callable_obj(*args, **kwargs)
OSError: [Errno 38] Function not implemented
"""

I wonder if we could add a @support.ignore_enosys decorator...

python-dev · 2013-10-25T06:31:29Z

New changeset 87d41a5a9077 by Christian Heimes in branch 'default':
Issue bpo-16595: prlimit() needs Linux kernel 2.6.36+
http://hg.python.org/cpython/rev/87d41a5a9077

tiran · 2013-10-25T06:33:23Z

The buildbot is Linux-2.6.35-vs2.3.0.36.32-gentoo-i686-Intel-R-Core-TM-2_CPU_6600@_2.40GHz-with-gentoo-2.1 but prlimit() requires 2.6.36+. I didn't expect to see a combination of glibc with prlimit() and Kernel without prlimit(). According to man prlimit it's not suppose to fail with ENOSYS, too.

vstinner · 2013-10-25T06:35:00Z

Or we should extend with supress(OSerror, errno=errno.ENOSYS): ... :-)

(Just kidding, ignored tests must be marked as skipped.)

neologix · 2013-10-25T13:13:36Z

I didn't expect to see a combination of glibc with prlimit() and Kernel without prlimit(). According to man prlimit it's not suppose to fail with ENOSYS, too.

Yeah, we've seen this several times on some buildbots.
Note that actually, any syscall can fail with ENOSYS: the configure
check just checks that the libc exposes the wrapper library, but when
the libc wrapper makes the actual syscall, the kernel can return
-ENOSYS if it doesn't implement the syscall.

tiran · 2013-11-20T16:45:33Z

Tests are passing on all buildbots for quite some time now.

giampaolo · 2013-12-10T18:27:42Z

Just received this report on psutil bug tracker:
https://code.google.com/p/psutil/issues/detail?id=455
It seems os.prlimit() is affected by the same problem:

>>> import resource
>>> resource.RLIM_INFINITY
-1
>>>

giampaolo · 2013-12-10T18:28:28Z

s/os.prlimit/resource.prlimit

pitrou · 2013-12-10T18:40:35Z

How is that a problem? In any case, this shouldn't have anything to do with prlimit(), please open another issue.

tiran self-assigned this Dec 2, 2012

tiran added extension-modules C modules in the Modules dir type-feature A feature request or enhancement labels Dec 2, 2012

tiran closed this as completed Nov 20, 2013

giampaolo reopened this Dec 10, 2013

pitrou closed this as completed Dec 10, 2013

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add resource.prlimit #60799

Add resource.prlimit #60799

tiran commented Dec 2, 2012

tiran commented Dec 2, 2012

tiran commented Dec 2, 2012

tiran commented Jun 24, 2013

tiran commented Jul 7, 2013

giampaolo commented Jul 19, 2013

vajrasky mannequin commented Jul 19, 2013

tiran commented Oct 21, 2013

python-dev mannequin commented Oct 22, 2013

tiran commented Oct 22, 2013

pitrou commented Oct 22, 2013

tiran commented Oct 22, 2013

neologix mannequin commented Oct 25, 2013

python-dev mannequin commented Oct 25, 2013

tiran commented Oct 25, 2013

vstinner commented Oct 25, 2013

neologix mannequin commented Oct 25, 2013

tiran commented Nov 20, 2013

giampaolo commented Dec 10, 2013

giampaolo commented Dec 10, 2013

pitrou commented Dec 10, 2013

Add resource.prlimit #60799

Add resource.prlimit #60799

Comments

tiran commented Dec 2, 2012

tiran commented Dec 2, 2012

tiran commented Dec 2, 2012

tiran commented Jun 24, 2013

tiran commented Jul 7, 2013

giampaolo commented Jul 19, 2013

vajrasky mannequin commented Jul 19, 2013

tiran commented Oct 21, 2013

python-dev mannequin commented Oct 22, 2013

tiran commented Oct 22, 2013

pitrou commented Oct 22, 2013

tiran commented Oct 22, 2013

neologix mannequin commented Oct 25, 2013

python-dev mannequin commented Oct 25, 2013

tiran commented Oct 25, 2013

vstinner commented Oct 25, 2013

neologix mannequin commented Oct 25, 2013

tiran commented Nov 20, 2013

giampaolo commented Dec 10, 2013

giampaolo commented Dec 10, 2013

pitrou commented Dec 10, 2013