compiler.compile fails on "if" statement in attached file

[menegazzobr]

BPO	14169
Nosy	@terryjreedy, @mdickinson, @benjaminp, @meadori, @JimJJewett
Files	small_with_error.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-03-10.16:30:09.549>
created_at = <Date 2012-03-01.18:03:05.382>
labels = ['type-bug']
title = 'compiler.compile fails on "if" statement in attached file'
updated_at = <Date 2012-03-10.16:30:09.547>
user = 'https://bugs.python.org/menegazzobr'

bugs.python.org fields:

activity = <Date 2012-03-10.16:30:09.547>
actor = 'terry.reedy'
assignee = 'none'
closed = True
closed_date = <Date 2012-03-10.16:30:09.549>
closer = 'terry.reedy'
components = []
creation = <Date 2012-03-01.18:03:05.382>
creator = 'menegazzobr'
dependencies = []
files = ['24699']
hgrepos = []
issue_num = 14169
keywords = []
message_count = 8.0
messages = ['154707', '154803', '155290', '155304', '155305', '155306', '155308', '155321']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'mark.dickinson', 'benjamin.peterson', 'meador.inge', 'Jim.Jewett', 'menegazzobr']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14169'
versions = ['Python 2.7']

[menegazzobr]

compiler.compile fails on "if" statement in attached file.

When executing the code

    compiler.compile(contents, '<string>', 'exec')

passing the attached file contents, the following error is raised:

ValueError: chr() arg not in range(256)

This won't fail when using the builtin "compile". Also removing the "if" statement or any line before it, the contents are compiled successfully.

[terryjreedy]

Please provide a *minimal* example that exhibits the problem. 'Minimal' means that if a single line is removed, the problem disappears.

[jimjjewett]

If I understood correctly, that *is* the minimal case, which suggests some sort of size problem.

That said, I could not duplicate on 2.6.2, nor with py_compile.compiler in 3.2.2; I am not currently sufficiently motivated to install another version just for this.

[mdickinson]

Here's the trackback I get with Python 2.7. Look's like something's assuming that POP_JUMP_IF_FALSE target of 65541 fits in 16 bits.

Python 2.7.2 (default, Jan 13 2012, 17:11:09) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> with open('small_with_error.py') as f:
...     contents = f.read()
... 
>>> import compiler
>>> compiler.compile(contents, '<string>', 'exec')
POP_JUMP_IF_FALSE 65541
114 5 256
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/pycodegen.py", line 65, in compile
    gen.compile()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/pycodegen.py", line 117, in compile
    self.code = gen.getCode()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/pycodegen.py", line 248, in getCode
    return self.graph.getCode()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/pyassem.py", line 313, in getCode
    self.makeByteCode()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/pyassem.py", line 518, in makeByteCode
    lnotab.addCode(self.opnum[opname], lo, hi)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/pyassem.py", line 612, in addCode
    self.code.append(chr(arg))
ValueError: chr() arg not in range(256)

[mdickinson]

Interestingly, dis.dis also fails to decompile the .pyc bytecode generated by importing the file. (This was after adding dummy AddUnit, MakeCustomaryToBase and MakeBaseToCustomary definitions to the top of the file to prevent the import raising an exception.)

>>> with open('small_with_error.pyc', 'r') as f:
...     bytecode = f.read()
... 
>>> bytecode = bytecode[8:]
>>> dis.dis(bytecode)

<... lots of output snipped ...>

       65686 LOAD_GLOBAL         4 (4)
       65689 STOP_CODE      
       65690 STOP_CODE      
       65691 POP_JUMP_IF_TRUE 27749
       65694 BUILD_TUPLE      1140
       65697 STOP_CODE      
       65698 STOP_CODE      
       65699 STOP_CODE      
       65700 STORE_GLOBAL    26482 (26482)
       65703 POP_JUMP_IF_TRUE  1652
       65706 STOP_CODE      
       65707 STOP_CODE      
       65708 STOP_CODE      
       65709 COMPARE_OP      24951
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/dis.py", line 45, in dis
    disassemble_string(x)
  File "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/dis.py", line 147, in disassemble_string
    print '(' + cmp_op[oparg] + ')',
IndexError: tuple index out of range

[mdickinson]

D'oh. Ignore that last message; all it shows is that I'm stupid.

[mdickinson]

Okay, so the issue is simply that the compiler module doesn't have logic for emitting the EXTENDED_ARGS[1] bytecode instruction to properly deal with a jump target that won't fit in two bytes. Given that the compiler module is deprecated, I propose closing this as 'won't fix'.

[1] http://docs.python.org/library/dis.html#opcode-EXTENDED_ARG

[terryjreedy]

I agree. 'deprecated' pretty much means "We do not think this is worth the effort of maintaining it. Don't use it unless it is already in old code."