sys.setfilesystemencoding("xxx"); open("a") => stack overflow

[vstinner]

BPO	8226
Nosy	@vstinner, @ned-deily
Files	setfilesystemencoding.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-03-25.00:36:34.321>
created_at = <Date 2010-03-24.23:05:53.096>
labels = ['interpreter-core', 'type-crash']
title = 'sys.setfilesystemencoding("xxx"); open("a") => stack overflow'
updated_at = <Date 2010-03-25.00:36:34.319>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2010-03-25.00:36:34.319>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2010-03-25.00:36:34.321>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2010-03-24.23:05:53.096>
creator = 'vstinner'
dependencies = []
files = ['16635']
hgrepos = []
issue_num = 8226
keywords = ['patch']
message_count = 4.0
messages = ['101656', '101657', '101658', '101662']
nosy_count = 2.0
nosy_names = ['vstinner', 'ned.deily']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue8226'
versions = ['Python 3.1', 'Python 3.2', 'Python 3.3']

[vstinner]

sys.setfilesystemencoding() doesn't check if the argument is a valid encoding name.

If the filesystem encoding is invalid, open("a") goes into an unlimited loop. The default recursion limit is 1000, but the example crash at 930: too bad :-) Each loop allocate ~9000 bytes: Linux creates a 8 MB stack by default, and so ~9000x930 uses all the stack.

Using a lower recursion limit, we can see the loop:
----------

$ ./python -c 'import sys; sys.setrecursionlimit(30); sys.setfilesystemencoding("xxx"); open("x")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 98, in search_function
    level=0)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 83, in search_function
    norm_encoding = normalize_encoding(encoding)
  File "/home/SHARE/SVN/py3k/Lib/encodings/__init__.py", line 55, in normalize_encoding
    if isinstance(encoding, bytes):
RuntimeError: maximum recursion depth exceeded while calling a Python object

---

[vstinner]

Attached patch raises a LookupError is the specified encoding is invalid. It uses _PyCodec_Lookup() function: is it allowed in bltinmodule.c?

[ned-deily]

(duplicate of bpo-8076)

[vstinner]

Commited: r79393 (py3k), r79394 (3.1).