compile of code with incorrect encoding yields MemoryError

[atuining]

BPO	979739
Nosy	@mwhudson, @theller
Files	thats-no-memory-error.diff: one approach.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2005-10-03.14:03:20.000>
created_at = <Date 2004-06-25.13:45:23.000>
labels = ['interpreter-core']
title = 'compile of code with incorrect encoding yields MemoryError'
updated_at = <Date 2005-10-03.14:03:20.000>
user = 'https://bugs.python.org/atuining'

bugs.python.org fields:

activity = <Date 2005-10-03.14:03:20.000>
actor = 'atuining'
assignee = 'nnorwitz'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2004-06-25.13:45:23.000>
creator = 'atuining'
dependencies = []
files = ['1322']
hgrepos = []
issue_num = 979739
keywords = []
message_count = 5.0
messages = ['21298', '21299', '21300', '21301', '21302']
nosy_count = 4.0
nosy_names = ['mwh', 'nnorwitz', 'theller', 'atuining']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue979739'
versions = ['Python 2.3']

[atuining]

The following code will fail in both Python 2.3 and
Python 2.4, raising a MemoryError exception, when run
on any platform except Windows:

compile("# -*- coding: mbcs -*-", "test.py", "exec")

This has been reproduced on the following platforms:

Linux x86
HP-UX
Solaris
Tru64 Unix

I realize this is an invalid encoding but it would be
nice if something other than MemoryError was raised!

[theller]

Logged In: YES
user_id=11105

I assume the behaviour occurrs when an unknown encoding is
specified. It can be reproduced on Windows:

compile("# -*- coding: xxx -*-", "test.py", "exec")

It should probably raise a SyntaxError, as trying to import
a module containing this encoding does.

The problem seems that when
PyParser_ParseStringFlagsFilename() calls
PyTokenizer_FromString(), and when the latter fails the
error is set to E_NOMEM.

[mwhudson]

Logged In: YES
user_id=6656

Here's a simple and seemingly effective patch. I'm not sure
it's "the right thing", though. The whole issue of whether
the parser should or should not set exceptions is a
frightful mess.

[nnorwitz]

Logged In: YES
user_id=33168

Duplicate of bpo-772896. This has been fixed in CVS. It's up
to Anthony whether it should be backported to 2.4 or not.

[atuining]

Logged In: YES
user_id=619560

No need to backport -- the solution is quite simple. I was
simply reporting it so that it would eventually get fixed.
And it has, so I'm happy. :-)