New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"--config python:foo" module search path #1349

Closed
dimaqq opened this Issue Sep 8, 2016 · 12 comments

Comments

5 participants
@dimaqq

dimaqq commented Sep 8, 2016

What I'm trying to do:

# myapp/gunicorn.py
from . import something

def on_starting(...): ...

If I try to run it with gunicorn -c file:myapp/gunicorn.py ... if fails with "cannot perform relative import" because this file is not loaded as a module. OK, I can accept that.

If I try to run it with gunicorn -c python:myapp.gunicorn ... it fails with "Error: no module named 'myapp'".

Digging deeper, I find that when I run python, I get following for sys.path:
['', '/usr/local/lib/python35.zip', '/usr/local/lib/python3.5', '/usr/local/lib/python3.5/plat-linux', '/usr/local/lib/python3.5/lib-dynload', '/usr/local/lib/python3.5/site-packages']

However, when I run gunicorn, it has following for sys.path:
['/usr/local/bin', '/usr/local/lib/python35.zip', '/usr/local/lib/python3.5', '/usr/local/lib/python3.5/plat-linux', '/usr/local/lib/python3.5/lib-dynload', '/usr/local/lib/python3.5/site-packages']

(nevermind system-wide paths, that's because docker)

What's the canonical way?

  • install myapp into system path?
  • teach gunicorn to add "." to system path? (is there a command-line argument?)
  • ugly hack in my config? (e.g. -c file:xxx which will chain-load real config)
@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented Oct 1, 2016

Usually I would recommend installing the app and gunicorn into a virtualenv. On docker, the virtualenv isn't necessary because the container provides the isolated environment. Note that PEP8 recommends always using absolute imports.

@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented Oct 1, 2016

You could probably also set the PYTHONPATH environment variable if you really don't want to install as a package.

@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented Oct 1, 2016

I wonder if Gunicorn should add the working directory to the python path, though. It might make sense.

@benoitc

This comment has been minimized.

Owner

benoitc commented Feb 26, 2017

@tilgovi what do you mean ? :)

@dimaqq

This comment has been minimized.

dimaqq commented Feb 26, 2017

OT: I'm not working on that product any more. In the end I worked around the limitation using a bootstrap file (i.e. start file:bootstrap.py which in turn sets up sys.path and imports real code). Doing that was also convenient for other settings (e.g. gunicorn settings).

@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented Feb 26, 2017

@benoitc looking at the paths in the example, only the first one is different.

Python has ''.
Gunicorn has '/usr/bin/python'

The current directory seems more appropriate than the location of the gunicorn script.

I'll have a look at what happens with other entry points style scripts and such. I'll see if there's a simple thing I can recommend.

@benoitc

This comment has been minimized.

Owner

benoitc commented Feb 26, 2017

@tilgovi maybe doing something like we do when the pythonpath setting is given?

if self.cfg.pythonpath and self.cfg.pythonpath is not None:
paths = self.cfg.pythonpath.split(",")
for path in paths:
pythonpath = os.path.abspath(path)
if pythonpath not in sys.path:
sys.path.insert(0, pythonpath)

though I noticed that we are already adding the current directory to the sys path:

sys.path.insert(0, self.cfg.chdir)

So not sure why it wouldn't be found. Otherwise we could simply add '' to the sys.path in any case

@tilgovi tilgovi self-assigned this Mar 10, 2017

tilgovi added a commit that referenced this issue Mar 12, 2017

Set CWD and Python path before and after config
The config may be specified as a Python module, in which case we want
to ensure that the Python path is fixed up properly before we try to
load it. That means we should follow symlinks and add the current working
directory before and after the configuration is loaded.

Fix #1349

tilgovi added a commit that referenced this issue Mar 13, 2017

Set CWD and Python path before and after config
The config may be specified as a Python module, in which case we want
to ensure that the Python path is fixed up properly before we try to
load it. That means we should follow symlinks and add the current working
directory before and after the configuration is loaded.

Fix #1349

tilgovi added a commit that referenced this issue Mar 14, 2017

Set CWD and Python path before and after config
The config may be specified as a Python module, in which case we want
to ensure that the Python path is fixed up properly before we try to
load it. That means we should follow symlinks and add the current working
directory before and after the configuration is loaded.

Fix #1349

tilgovi added a commit that referenced this issue Oct 27, 2017

Set CWD and Python path before and after config
The config may be specified as a Python module, in which case we want
to ensure that the Python path is fixed up properly before we try to
load it. That means we should follow symlinks and add the current working
directory before and after the configuration is loaded.

Fix #1349
@mindojo-victor

This comment has been minimized.

mindojo-victor commented Dec 30, 2017

Was there a release with these changes?

@benoitc

This comment has been minimized.

Owner

benoitc commented Dec 30, 2017

fofanov pushed a commit to fofanov/gunicorn that referenced this issue Mar 16, 2018

Set CWD and Python path before and after config
The config may be specified as a Python module, in which case we want
to ensure that the Python path is fixed up properly before we try to
load it. That means we should follow symlinks and add the current working
directory before and after the configuration is loaded.

Fix benoitc#1349
@YunLiaw

This comment has been minimized.

YunLiaw commented Mar 20, 2018

Hi @benoitc, is there any plan for releasing this feature? Thanks!

@tilgovi

This comment has been minimized.

Collaborator

tilgovi commented Apr 28, 2018

@YunLiaw 19.8.0 is released!

@YunLiaw

This comment has been minimized.

YunLiaw commented Apr 28, 2018

@tilgovi Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment