New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible for env.py to be env.pyc? #449

Closed
sqlalchemy-bot opened this Issue Sep 7, 2017 · 16 comments

Comments

Projects
None yet
1 participant
@sqlalchemy-bot

sqlalchemy-bot commented Sep 7, 2017

Migrated issue, originally created by Cody Scott (@Siecje)

I'm running into a problem with alembic. I don't have a env.py only a env.pyc and this line is failing. https://bitbucket.org/zzzeek/alembic/src/4effc8124fa2ad5aab42119e4e7aa1452982d257/alembic/script/base.py?at=master&fileviewer=file-view-default#base.py-425

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Michael Bayer (@zzzeek) wrote:

no issue here. Please define "failing" with complete information, thanks.

[classic@photon2 alembic]$ PYTHONPATH=~/dev/sqlalchemy/lib/ python -m alembic.config current
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
e3f2bd11c8f9 (head)
[classic@photon2 alembic]$ rm foo/^C
[classic@photon2 alembic]$ mv foo/env.py foo/env.py.deleted
[classic@photon2 alembic]$ ls foo/
env.pyc  env.py.deleted  README  script.py.mako  versions
[classic@photon2 alembic]$ PYTHONPATH=~/dev/sqlalchemy/lib/ python -m alembic.config current
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
e3f2bd11c8f9 (head)

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

Yes sourceless was set to true, which allowed the versions to be loaded.

This is the traceback that I'm getting.

I'm trying to run jupyterhub which uses alembic.

    Traceback (most recent call last):
      File "/tmp/pip-build-1zvsjpso/jupyterhub/jupyterhub/app.py", line 1623, in launch_instance_async
      File "/boot/tp/python3/lib/python3.6/types.py", line 204, in __next__
        return next(self.__wrapped)
      File "/tmp/pip-build-1zvsjpso/jupyterhub/jupyterhub/app.py", line 1376, in initialize
      File "/tmp/pip-build-1zvsjpso/jupyterhub/jupyterhub/app.py", line 900, in init_db
      File "/tmp/pip-build-1zvsjpso/jupyterhub/jupyterhub/orm.py", line 530, in new_session_factory
      File "/tmp/pip-build-1zvsjpso/jupyterhub/jupyterhub/orm.py", line 475, in check_db_revision
      File "/boot/forge/pypi_jupyterhub/build/lib/python3.6/site-packages/alembic/command.py", line 496, in stamp
      File "/boot/forge/pypi_jupyterhub/build/lib/python3.6/site-packages/alembic/script/base.py", line 416, in run_env
      File "/boot/forge/pypi_jupyterhub/build/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 99, in load_python_file
    ImportError: Can't find Python file /boot/tp/python3/lib/python3.6/site-packages/jupyterhub/alembic/env.py
@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

admin@hostname-not-set:/mnt/vda1/tp/python3/lib/python3.6/site-packages/jupyterhub/alembic$ ls 
README          env.pyc         script.py.mako  versions/
admin@hostname-not-set:/mnt/vda1/tp/python3/lib/python3.6/site-packages/jupyterhub/alembic$ pwd
/boot/tp/python3/lib/python3.6/site-packages/jupyterhub/alembic
@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

I think I figured it out.
I'm using Python3.6 and PYTHONOPTIMIZE is set so

sys.flags.optimize is 1.

simple_pyc_file_from_path is adding a o to the path.

In Python3 there are no more .pyo files as per PEP 488.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Michael Bayer (@zzzeek) wrote:

OK...but is your .pyc file in the local path where the .py would be? or is it in a __pycache__ ?

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Michael Bayer (@zzzeek) wrote:

no more .pyo files as of python 3.5, not python 3

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

It is in the local path where the .py file would be.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Michael Bayer (@zzzeek) wrote:

how are you generating that .pyc, since on Python 3.5+, if you do py_compile(path), the file goes into pycache with a special name? I'm just trying to understand how folks even make these sourceless installs. i have most of a fix for all of this but it still needs the version files, pyc or whatever, to be locally present in versions/ and not in a pycache.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Michael Bayer (@zzzeek) wrote:

though I can make it work that way...

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

After they are compiled I move them up out of the __pycache__ folder and rename the file to just module.pyc, so that modules are where imports expect them.

Source

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

In versions/ I just have the .pyc files, no __pycache__, which works when I either restore env.py or copy env.pyc to env.pyo.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 7, 2017

Cody Scott (@Siecje) wrote:

Wow that was a lot bigger than I thought. Yes that works!

Thanks for spending so much time on this.

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 8, 2017

Michael Bayer (@zzzeek) wrote:

Support pep3149, latest import mechanics, fully

Reworked "sourceless" system to be fully capable of handling any
combination of: Python2/3x, pep3149 or not, PYTHONOPTIMIZE or not,
for locating and loading both env.py files as well as versioning files.
This includes: locating files inside of __pycache__ as well as listing
out version files that might be only in versions/__pycache__, deduplicating
version files that may be in versions/__pycache__ and versions/
at the same time, correctly looking for .pyc or .pyo files based on
if pep488 is present or not. The latest Python3x deprecation warnings
involving importlib are also corrected.

Change-Id: I2495e793c81846d3f05620dbececb18973dd8a8f
Fixes: #449

9b5f12a

@sqlalchemy-bot

This comment has been minimized.

sqlalchemy-bot commented Sep 8, 2017

Changes by Michael Bayer (@zzzeek):

  • changed status to closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment