pex --python seems to ignore the selected interpreter path #53
Comments
|
Yeah, hard to tell if this is a feature or a bug. The way pex treats interpreters is as "interpreter classes" -- a tuple of flavor (CPython, PyPy) and major/minor version (2.6, 2.7, 3.4.) So when you do --python= it uses that interpreter to build dependencies but then synthesizes the shebang line based upon the interpreter class instead of as the explicit interpreter. This is mostly for pex portability sake, since '#!/home/wickman/herp/derp/python2.7" is usually not what you want (but admittedly, it could be useful.) Right now PEX_INTERPRETER is just a boolean to determine whether or not to drop into a repl. I'd like to change that to PEX_REPL=1 and use PEX_INTERPRETER the same way as --python. In other words, if it's an explicit path, re-exec to that interpreter, otherwise, try and search $PATH for an interpreter that satisfies a requirement e.g. PEX_INTERPRETER='PyPy>=2.4,<2.5'. This is why the PythonInterpreter.replace method exists but it has not yet been used in practice. |
|
Just to provide a use-case: I'm building a pex that will then get further bundled into a distributable format for Ubuntu Core Snappy. In that execution environment, there may not be a suitable $PATH to resolve the shebang, but I know that /usr/bin/python3 (or /usr/bin/python3.4) will exist. So I need a way to tell pex "no, really, this is the exact path I want you to use for the shebang". It's tricky to munge the shebang after the pex has been written. |
|
Totally agree this makes sense. I don't want it to be the default behavior but we should allow this. It should probably be an option to PythonInterpreter (e.g. preserve_hashbang=True?) but not completely sure what the API should look like. Thoughts? |
|
Looking at just the --help output, I found it surprising that --python didn't preserve my argument verbatim. I'm not sure if you've been following PEP 441, but we had a similar discussion over there. Totally get the need for pex to preserve its backward compatibility of course. I suppose in that case, something like |
|
Good point. I thought about it a little more and I thought you're right, --python should probably be 'use this interpreter, exactly.' Perhaps a separate command flag, --python-version, could be switched out with the current --python semantics. |
|
On Feb 20, 2015, at 02:39 PM, brian wickman wrote:
+1 :) |
* Adds ``--python-shebang`` option to the pex tool in order to set the ``#!`` shebang to an exact path. * Adds support for ``PEX_PYTHON`` environment variable which will cause the pex file to reinvoke itself using the interpreter specified, e.g. ``PEX_PYTHON=python3.4`` or ``PEX_PYTHON=/exact/path/to/interpreter``.
|
I ultimately decided to leave --python alone. However, I added --python-shebang which allows you to set the shebang line exactly. I also added the PEX_PYTHON environment variable which allows you to change which interpreter invokes the PEX, e.g. mba=pex=; head -1 dist/pex
#!/usr/bin/env python2.7
mba=pex=; PEX_PYTHON=pypy PEX_INTERPRETER=1 dist/pex
Python 2.7.8 (f5dcc2477b97, Sep 19 2014, 18:09:54)
[PyPy 2.4.0 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> This is on master now with 1.0.0.dev3. 'tox -e py34-package' to build a pex with all the above functionality. |
The pex_binary interpreter attribute is documented as "Path to the python interpreter the pex should to use in its shebang line." However, it was not being passed when building the PEX, so it did not take effect. It is getting passed as PEX_PYTHON, which changes the interpreter used for *that invocation* of a given pex. This may have been a recent change: pex-tool/pex#53 interpreter_test.py: Add a test that reads the files and verifies that the interpreter is set correctly.
The pex_binary interpreter attribute is documented as "Path to the python interpreter the pex should to use in its shebang line." However, it was not being passed when building the PEX, so it did not take effect. It is getting passed as PEX_PYTHON, which changes the interpreter used for *that invocation* of a given pex. This may have been a recent change: pex-tool/pex#53 interpreter_test.py: Add a test that reads the files and verifies that the interpreter is set correctly.
It doesn't seem to matter if
--python=/usr/bin/python3.4is used (i.e. it's not related to python3 being a symlink).The text was updated successfully, but these errors were encountered: