Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-34726: Fix handling of hash-based pycs in zipimport #10327

Merged
merged 3 commits into from Nov 7, 2018

Conversation

elprans
Copy link
Contributor

@elprans elprans commented Nov 4, 2018

Current support for hash-based bytecode files in zipimport is rather
sparse, which leads to test failures when the test suite is ran with
the SOURCE_DATE_EPOCH environment variable set.

This teaches zipimport to handle hash-based pycs properly.

https://bugs.python.org/issue34022

https://bugs.python.org/issue34726

Current support for hash-based bytecode files in `zipimport` is rather
sparse, which leads to test failures when the test suite is ran with
the ``SOURCE_DATE_EPOCH`` environment variable set.

This teaches zipimport to handle hash-based pycs properly.
Copy link
Contributor

@ncoghlan ncoghlan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests look good to me, and the changes required to make them pass seem sensible.

@ncoghlan
Copy link
Contributor

ncoghlan commented Nov 5, 2018

Thanks for the PR @elprans!

@serhiy-storchaka @warsaw @brettcannon This looks good to me, but I'd be happier if someone more familiar with the new zipimport code gave it a look before we hit the big green merge button :)

@serhiy-storchaka serhiy-storchaka changed the title bpo-34022: Fix handling of hash-based pycs in zipimport bpo-34726: Fix handling of hash-based pycs in zipimport Nov 5, 2018
@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Nov 5, 2018

I created bpo-34726, but I'm not sure that any checks for pyc files are needed at all. They slow down import, and we can expect that pyc files in a zip archive are up to date with py files. See bpo-23734.

Lib/zipimport.py Outdated Show resolved Hide resolved
Lib/zipimport.py Outdated Show resolved Hide resolved
@serhiy-storchaka
Copy link
Member

You need to rebuild frozen modules. Run make regen-importlib.

@elprans
Copy link
Contributor Author

elprans commented Nov 7, 2018

Done

@serhiy-storchaka serhiy-storchaka merged commit a6e956b into python:master Nov 7, 2018
yan12125 pushed a commit to archlinuxcn/repo that referenced this pull request Nov 12, 2018
@vstinner
Copy link
Member

test_zipfile_compiled_checked_hash() fails on Python 3.7: we should backport this bugfix to 3.7, no?

@miss-islington
Copy link
Contributor

Thanks @elprans for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry, @elprans and @serhiy-storchaka, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker a6e956bcb0edbfe7f18af9be2215a5326ea6bf05 3.7

@vstinner
Copy link
Member

Ah, the change cannot be cherry-picked automatically. Maybe because of the generated file.

@elprans
Copy link
Contributor Author

elprans commented Nov 12, 2018

@vstinner I'll make a backport PR.

@elprans
Copy link
Contributor Author

elprans commented Nov 13, 2018

It's a bit more difficult than that. In 3.7 zipimport is in C.

@serhiy-storchaka
Copy link
Member

It is easier to skip the test on 3.7.

@serhiy-storchaka serhiy-storchaka removed their assignment Nov 13, 2018
dundee pushed a commit to dundee/pkgbuilds that referenced this pull request Feb 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants