Remove inconsistent behavior between import and zipimport

[acidbase]

BPO	1346572
Nosy	@loewis, @brettcannon, @ncoghlan, @merwok
Files	patch_import.diff: Patch to 'fix' this inconsitent behavior import/zipimport

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-04-22.17:14:45.016>
created_at = <Date 2005-11-03.03:43:17.000>
labels = ['interpreter-core']
title = 'Remove inconsistent behavior between import and zipimport'
updated_at = <Date 2012-04-22.17:14:45.012>
user = 'https://bugs.python.org/acidbase'

bugs.python.org fields:

activity = <Date 2012-04-22.17:14:45.012>
actor = 'eric.araujo'
assignee = 'none'
closed = True
closed_date = <Date 2012-04-22.17:14:45.016>
closer = 'eric.araujo'
components = ['Interpreter Core']
creation = <Date 2005-11-03.03:43:17.000>
creator = 'acidbase'
dependencies = []
files = ['6851']
hgrepos = []
issue_num = 1346572
keywords = ['patch']
message_count = 11.0
messages = ['48962', '48963', '154681', '154682', '154698', '154703', '154711', '154733', '154738', '154739', '154741']
nosy_count = 5.0
nosy_names = ['loewis', 'brett.cannon', 'ncoghlan', 'acidbase', 'eric.araujo']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1346572'
versions = []

[acidbase]

Look the inconsistent behavior:
$ ls
modulo_c.pyc modulo_o.pyo
$ zip modulos.zip modulo_o.pyo modulo_c.pyc
adding: modulo_o.pyo (deflated 38%)
adding: modulo_c.pyc (deflated 38%)
$ ls
modulo_c.pyc modulo_o.pyo modulos.zip

$ python2.4
>>> import modulo_c
modulo_c
>>> import modulo_o
ImportError: No module named modulo_o

$ python2.4 -O
>>> import modulo_c
ImportError: No module named modulo_c
>>> import modulo_o
modulo_o

$ rm *.pyc *.pyo
$ ls
modulos.zip
$ PYTHONPATH=modulos.zip python2.4
>>> import modulo_c
modulo_c
>>> import modulo_o
modulo_o
$ PYTHONPATH=modulos.zip python2.4 -O
>>> import modulo_c
modulo_c
>>> import modulo_o
modulo_o

[loewis]

Logged In: YES
user_id=21627

The discussion on python-dev (both last year's and this
year's) has shown that this is the wrong way of fixing the
inconsistency. Instead, zipfile should stop importing .pyo
files when -O isn't given. Rejecting the patch as invalid.

[merwok]

Still an issue with 3.2: zipimport considers both pyc and pyo with or without -O.

The discussion starts here: http://mail.python.org/pipermail/python-dev/2006-November/thread.html#69822

(This was originally closed instead of asking for another fix because SourceForge separated bugs and patches.)

[merwok]

I need to qualify something: I reproduced the bug with legacy/pre-PEP 3147/in-the-same-dir pyc and pyo files. One could argue that pycache directories in 3.2+ make this irrelevant and that it’s too late for 2.x.

[brettcannon]

I like that argument. =) If this is not an issue in Python 3.3 then this should be closed as out of date since it will break code if it is changed.

[merwok]

It entirely depends on how much you care about pyc-only/pyo-only zipfile distributions, and compatibility between zipimport and importlib (i.e. if you don’t plan on matching the zipimport bug in importlib, might as well fix zipimport in 2.7 and 3.2).

[brettcannon]

I don't care about compatibility between zipimport and importlib.

[merwok]

OK, I’m leaving this open until the next weekly report just in case someone interested comes here and weighs in, otherwise I’ll close as wontfix.

[ncoghlan]

Does "compileall" generate both .pyc and .pyo by default? Or do you have to run it twice? If the latter, does pysetup handle that for you?

MvL is correct that zipimport should ignore .pyo files when __debug__ is set and vice-versa, but the precompilation tools should also take care of generating both versions by default.

[merwok]

Does "compileall" generate both .pyc and .pyo by default?
python3 -m compileall generates pyc, python3 -O -m compileall pyo. Functions in py_compile and compileall gained an optimize argument in 3.2.

does pysetup handle that for you?
You’ll have to be more specific. Bytecode files can be created when building a bdist or on install from source. Packaging commands do not depend on the calling Python’s -O option, they have their own options to let users specify if they want pyc files, pyo, neither or both.

MvL is correct that zipimport should ignore .pyo files when __debug__ is set and vice-versa, but the
precompilation tools should also take care of generating both versions by default.
Really? The behaviors of compileall and packaging seems better to me.

[ncoghlan]

Status quo sounds fine then. +1 for closing it again.