IDLE not displaying RecursionError tracebacks and hangs #70993
Comments
|
Test program: import sys
sys.setrecursionlimit(20)
def f(): return f()
f()F:\Python\mypy>python tem.py
Traceback (most recent call last):
File "tem.py", line 4, in <module>
f()
File "tem.py", line 3, in f
def f(): return f()
...
RecursionError: maximum recursion depth exceededIn 2.7.11, the error is caught and the user process restarted. ======================= RESTART: F:\Python\mypy\tem.py ======================= =============================== RESTART: Shell ===============================
In 3.5.1, the user process hangs, ^C does not work, and a restart explicitly requested either with Shell => Restart or running another program. This behavior is either peculiar to this test case, or a regression, as I remember getting proper RecursionError tracebacks in the past. |
|
In 3.6 just setting the recursion level to 20 produces following output on terminal: ---------------------------------------- and a hang. In 2.7 it produces: ---------------------------------------- Unhandled server exception!
Thread: SockThread
Client Address: ('127.0.0.1', 35043)
Request: <socket._socketobject object at 0xb70e4a84>
Traceback (most recent call last):
File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 290, in _handle_request_noblock
self.process_request(request, client_address)
File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 318, in process_request
self.finish_request(request, client_address)
File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 331, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/rpc.py", line 500, in __init__
SocketServer.BaseRequestHandler.__init__(self, sock, addr, svr)
File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 652, in __init__
self.handle()
File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/run.py", line 292, in handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/rpc.py", line 280, in getresponse
response = self._getresponse(myseq, wait)
File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/rpc.py", line 300, in _getresponse
response = self.pollresponse(myseq, wait)
RuntimeError: maximum recursion depth exceeded*** Unrecoverable, server exiting! and restarts the shell. Definitely the 20 limit is too low for IDLE. |
|
Experimentally found that minimal recursion level for 2.7 is 29, for 3.2-3.3 is 24, and for 3.4-3.6 is 25. 3.2 produces following output and restart the shell: ---------------------------------------- Unhandled server exception!
Thread: SockThread
Client Address: ('127.0.0.1', 37227)
Request: <socket.socket object, fd=3, family=2, type=1, proto=0>
Traceback (most recent call last):
File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 321, in process_request
self.finish_request(request, client_address)
File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 503, in __init__
socketserver.BaseRequestHandler.__init__(self, sock, addr, svr)
File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 648, in __init__
self.handle()
File "/home/serhiy/py/cpython-3.2/Lib/idlelib/run.py", line 285, in handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 280, in getresponse
response = self._getresponse(myseq, wait)
File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 300, in _getresponse
response = self.pollresponse(myseq, wait)
File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 421, in pollresponse
self.putmessage(message)
File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 324, in putmessage
s = pickle.dumps(message)
RuntimeError: maximum recursion depth exceeded while pickling an object*** Unrecoverable, server exiting! 3.3 hangs without any terminal output. 3.4 produces the most detailed output and hangs: ---------------------------------------- Unhandled server exception!
Thread: SockThread
Client Address: ('127.0.0.1', 46394)
Request: <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 48776), raddr=('127.0.0.1', 46394)>
Traceback (most recent call last):
Exception in thread SockThread:
Traceback (most recent call last):
File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 305, in _handle_request_noblock
self.process_request(request, client_address)
File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 331, in process_request
self.finish_request(request, client_address)
File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 344, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 508, in __init__
socketserver.BaseRequestHandler.__init__(self, sock, addr, svr)
File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 673, in __init__
self.handle()
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/run.py", line 318, in handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 288, in getresponse
response = self._getresponse(myseq, wait)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 308, in _getresponse
response = self.pollresponse(myseq, wait)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 426, in pollresponse
self.putmessage(message)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 332, in putmessage
s = dumps(message)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 60, in dumps
p.dump(obj)
RuntimeError: maximum recursion depth exceeded while pickling an object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/serhiy/py/cpython-3.4/Lib/threading.py", line 911, in _bootstrap_inner
self.run()
File "/home/serhiy/py/cpython-3.4/Lib/threading.py", line 859, in run
self._target(*self._args, **self._kwargs)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/run.py", line 162, in manage_socket
server.handle_request() # A single request only
File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 290, in handle_request
self._handle_request_noblock()
File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 307, in _handle_request_noblock
self.handle_error(request, client_address)
File "/home/serhiy/py/cpython-3.4/Lib/idlelib/run.py", line 288, in handle_error
traceback.print_exc(file=erf)
File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 252, in print_exc
print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 169, in print_exception
for line in _format_exception_iter(etype, value, tb, limit, chain):
File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 153, in _format_exception_iter
yield from _format_list_iter(_extract_tb_iter(tb, limit=limit))
File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 18, in _format_list_iter
for filename, lineno, name, line in extracted_list:
File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 65, in _extract_tb_or_stack_iter
line = linecache.getline(filename, lineno, f.f_globals)
File "/home/serhiy/py/cpython-3.4/Lib/linecache.py", line 15, in getline
lines = getlines(filename, module_globals)
File "/home/serhiy/py/cpython-3.4/Lib/linecache.py", line 42, in getlines
return updatecache(filename, module_globals)
File "/home/serhiy/py/cpython-3.4/Lib/linecache.py", line 130, in updatecache
with tokenize.open(fullname) as fp:
File "/home/serhiy/py/cpython-3.4/Lib/tokenize.py", line 458, in open
text = TextIOWrapper(buffer, encoding, line_buffering=True)
RuntimeError: maximum recursion depth exceeded3.5 and 3.6 behaves as was described in msg263795. |
|
Thanks for the investigation. I copied the limit from a python-ideas example and did not realize that IDLE adds so much to the call stack. I should add a bit on the subject to the IDLE doc in "3.2. IDLE-console differences". (It can also impact tracebacks -- as reported on a couple of tracker issues.) "IDLE adds it own calls to the call stack before (and possibly after) running user code. Tracebacks are filtered to remove these, but the filtering may not be perfect. Setting a recursion limit that is too low, such as with sys.setrecursionlimit(20), will result in mysterious malfunctions." |
|
We can monkeypatch sys.setrecursionlimit() to set recursion limit to at least say 50. |
|
Interesting idea. We avoid the problem by adding something like the following to run.py _setrecursionlimit = sys.setrecursionlimit
def setrecursionlimit(n):
_setrecursionlimit(max(n, 50))
sys.setrecursionlimit = setrecursionlimitThis works when entered interactively, and I presume it would within run.py. For _setrecursionlimit to be accessible from user code (to reverse the monkey patching), it would have to be attached to the function as an attribute. setrecursionlimit.original = _setrecursionlimitThough user-friendly for most users, this would make IDLE execute code differently from raw Python. The builtin has a lower limit, based on the current stack depth, but it raises instead of setting a higher limit. I presume we could use len(inspect.stack()) to get the current 'recursion depth', and add, say, 30 rather than 3. (The current error message could be more helpful.) >>> sys.setrecursionlimit(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
RecursionError: cannot set the recursion limit to 3 at the recursion depth 1: the limit is too low
>>> sys.setrecursionlimit(4)
>>> f()
>>>
>>> def f():
... print('f')
... f()
...
>>> f()
>>>The call to f seems to be silently ignored. This does not seem helpful. Whatever we do, I would mention it in a revision of the proposed paragraph above. |
|
I ran into this again. Raising priority. |
terryjreedy
changed the title
taleinat
added
3.8
|
IMO just updating the documentation would not be enough. I suggest wrapping sys.setrecursionlimit() and *adding* 30 to the given limit, to offset the call stack depth added by IDLE. This makes more sense to me than max(n, 50), which I imagine would cause a lot of confusion in cases where it comes into effect. If that's not acceptable, then I suggest wrapping it to raise an informative exception if the given limit is too low. |
|
'max(n, 50)' should have been 'min(n, 50)' Tal, does you comment avove about the former apply when corrected to the latter? Both get... and setrecursionlimit should be wrapped in a function, such as 'wrap_recursionlimits', called from run.main, each decorated with functools.wraps. Both docstrings should be augmented with ... .__doc__ += "\n\nWhen running under IDLE, ..." This is all that is needed for getrecursionlimit. The question is to do with setrecursionlimit. Whatever is chosen should be added to the IDLE doc section mentioned above. Currently, at least on CPython, n must be >= 1 or ValueError is raised. I don't like changing that. I don't have a strong feeling about threshholding versus incrementing, but see my question above. |
Yes. To use your terms, I prefer incrementing over thresholding.
If we go with incrementing, then getrecursionlimit should likely decrement the returned value by the same amount, so that setrecursionlimit(getrecursionlimit()) doesn't change the recursion limit.
This is another issue that incrementing avoids. Since it looks like we're in general agreement, Terry, I'll work up a PR. |
|
Setting the recursion limit too low also causes issues in a plain Python interactive session. With current master (to be 3.9) on Win10, I get issues with values up to 6. In IDLE values up to 20 cause an immediate exception similar to the one described by Serhiy. Anyways, I think we should increment by 25, to always set "safe" values. |
|
See PR #58152. |
|
[Initially written before you posted the PR] I agree that "get...(set...(n)) == n" (when n is positive, a check we should emulate) and "set...(get...())" leaving limit as it was are desirable. The docs and docstrings just need to say what we do. Possible additions. The main reason I can think of for users to play with low recursion limits is to explore the implementation. They simply cannot get stock CPython behavior when running with IDLE. So I want the IDLE wrappers to not be completely transparent. For that to be obvious in calltips, the first line should say so. The main reason I want to intervene is to prevent the not-nice behavior of hanging until a manual restart. I checked, and it is still possible. In 3.8.0b1, the min n is not 1 but current stack + 5.
>>> sys.setrecursionlimit(11)
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
sys.setrecursionlimit(11)
RecursionError: cannot set the recursion limit to 11 at the recursion depth 6: the limit is too lowOur increment will prevent a user from seeing this, but this is necessary to prevent ... >>> sys.setrecursionlimit(12)
# hangs, once after printing "Exception ignored in thread started by"
# until manual restart.
# 18 hangs after printing 60 lines of a double exception.
# 20 does not hang and calling 'f' from the original message
# gets a truncated traceback ending with
[Previous line repeated 12 more times] |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: