Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.7] bpo-25083: Python 2 can sometimes create incorrect .pyc files #8449

Merged
merged 1 commit into from
Sep 10, 2018

Conversation

tzickel
Copy link
Contributor

@tzickel tzickel commented Jul 24, 2018

Python 2 never checked for I/O error when reading .py files and
thus could mistake an I/O error for EOF and create incorrect .pyc
files. This adds an check for this and aborts on an error.

https://bugs.python.org/issue25083

@tzickel
Copy link
Contributor Author

tzickel commented Jul 24, 2018

backport-pr is failing, because this is a Python 2 only issue.

@methane methane changed the title bpo-25083: Python 2 can sometimes create incorrect .pyc files [2.7] bpo-25083: Python 2 can sometimes create incorrect .pyc files Jul 24, 2018
@encukou
Copy link
Member

encukou commented Jul 28, 2018

This looks good except for the use of E_EOF. That should be reserved for end-of-file, but in this case here we're not getting to the end of file.
I don't see a good E_ constant for read errors in the current list. Could you add a new one in Include/errcode.h and Python/pythonrun.c, so that this gets a proper error message?

@encukou
Copy link
Member

encukou commented Jul 28, 2018

And don't worry about the backport title check; that's for the person who'll merge this.

@tzickel
Copy link
Contributor Author

tzickel commented Jul 29, 2018

Done.

@tzickel
Copy link
Contributor Author

tzickel commented Jul 29, 2018

Here is a script to check for the bug:
https://gist.github.com/tzickel/2ab09100ea21ca0e1e6709c337194bff
needs a recent strace (ubuntu 18.04 or debian sed is fine), and if run inside a docker, the docker needs to be run with --cap-add SYS_PTRACE for strace to work.

@@ -1681,6 +1681,10 @@ int
PyTokenizer_Get(struct tok_state *tok, char **p_start, char **p_end)
{
int result = tok_get(tok, p_start, p_end);
if (tok->fp && ferror(tok->fp)) {
result = ERRORTOKEN;
tok->done = E_IO;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, should we call clearerr() here? In case if fp is used again.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Python 2 never checked for I/O error when reading .py files and
thus could mistake an I/O error for EOF and create incorrect .pyc
files. This adds an check for this and aborts on an error.
@tzickel
Copy link
Contributor Author

tzickel commented Aug 1, 2018

I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@vstinner: please review the changes made to this pull request.

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I would prefer that someone else merge it.

@tzickel
Copy link
Contributor Author

tzickel commented Sep 3, 2018

@encukou I'm waiting for the convincing part :) I think it's best to merge this ASAP (if it's plan to be used), so it has maximum exposure before a new release.

@encukou encukou closed this Sep 10, 2018
@encukou encukou reopened this Sep 10, 2018
@encukou encukou merged commit f64c813 into python:2.7 Sep 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants