Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Unable to execute compiled script with JEP and Python 3.5.3 #77
I have implemented a compile step to convert .py to .pyc files prior to execution. Each script is cached and if it is present it is used instead of the interpreted .py.
Everything works great when JEP is asked to execute the .pyc when on Python 2.7.13. However when I change environment and use Python 3.5.3 I get the following error...
Bad code object in .pyc file
Note I am aware of pyc magic numbers - the cached pyc files are cleaned and compiled with 3.5.3. Looking at pyembed.c I am getting past the magic number validation at line 1328 and seeing the error at the validation step at line 1334.
Here's the bit I can't understand. If I execute the compiled .pyc file using python3 directly it runs just fine.
I have attached a simple example that highlights the problem, the zip contains the following...
jepHello.py - native script
Please let me know if there's something I'm missing or if this is a bug and needs more investigation I'd be very happy to help. JEP is an awesome piece of work!
Incidentally I am using JEP as the caching compiler too using the following ...
This works flawlessly and the JEP generated .pyc runs fine on any version of native python.
Thanks for any help you can offer...
I am not very familiar with this code but I think I might have it figured out. The comment on pyembed_run_pyc makes it clear that the code was gratuitously copyied from pythonrun.c::run_pyc_file. I compared the code in run_pyc_file in python 2.7 with the code in run_pyc_file in python 3.5. It is clear that the python source has changed to read an extra long from the file before reading the code object. So my assumption is that we also need to read another long. Since we are supporting multiple python versions we will need some sort of "#if PY_MAJOR_VERSION..." to ensure we are compatible with both. It looks like the extra long was added in this change, which is python 3.3, so we should only read an extra long in versions 3.3 or newer.
@pjr1060 Can you try adding another PyMarshal_ReadLongFromFile in pyembed_run_pyc and see if that resolves the issue on python 3.5.3?
Hi Nathan - I just read your comment re: 3.7 release plan in issue #79
If there is to be a 3.6 update would it be possible to apply this fix to 3.6 for general release? Thanks again for all your help!