Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PATH modification is not passed to the test commands #1423

Closed
aklajnert opened this issue Sep 18, 2019 · 4 comments · Fixed by #1563
Closed

PATH modification is not passed to the test commands #1423

aklajnert opened this issue Sep 18, 2019 · 4 comments · Fixed by #1563
Labels
bug:normal pr-available

Comments

@aklajnert
Copy link

@aklajnert aklajnert commented Sep 18, 2019

Here is my example tox.ini:

[testenv]
skip_install = true
setenv =
  PATH = /path/to/binary
  TEST = test
whitelist_externals = echo
commands =
  python -c "import os; print(os.environ.get('PATH')); print(os.environ.get('TEST'))"
  echo {env:PATH}

I was expecting, that the os.environ.get('PATH'), will give the same result as echo {env:PATH}. Turns out it doesn't:

.tox finish: provision  after 8.63 seconds
[532] lvlogina::/tmp/tox-test > py -3.7 --aux -m tox -vv
using tox.ini: /tmp/tox-test/tox.ini (pid 21471)
could not satisfy requires DistributionNotFound(Requirement.parse('tox>=3.8.6'), None)
  removing /tmp/tox-test/.tox/log
using tox-3.8.6 from /local/method/modules/python/auxiliary/tox/__init__.py (pid 21471)
.tox start: getenv /tmp/tox-test/.tox/.tox
.tox reusing: /tmp/tox-test/.tox/.tox
.tox finish: getenv /tmp/tox-test/.tox/.tox after 0.13 seconds
.tox start: finishvenv
.tox finish: finishvenv  after 0.00 seconds
.tox start: provision
[21490] /tmp/tox-test$ /tmp/tox-test/.tox/.tox/bin/python -m tox -vv
using tox.ini: /tmp/tox-test/tox.ini (pid 21490)
  removing /tmp/tox-test/.tox/log
using tox-3.8.6 from /local/method/modules/python/auxiliary/tox/__init__.py (pid 21490)
python start: getenv /tmp/tox-test/.tox/python
python reusing: /tmp/tox-test/.tox/python
python finish: getenv /tmp/tox-test/.tox/python after 0.10 seconds
python start: finishvenv
python finish: finishvenv  after 0.00 seconds
python start: envreport
setting PATH=/tmp/tox-test/.tox/python/bin:/grid/common/bin:/local/method/infrastructure/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/sbin:/grid/sfi/hpc/pbspro/v18.1.2p1/wrappers/bin:/grid/scp/hw/apps/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/local/method/bin
[21510] /tmp/tox-test$ /tmp/tox-test/.tox/python/bin/python -m pip freeze >.tox/python/log/python-1.log
python finish: envreport  after 0.67 seconds
python installed: atomicwrites==1.3.0,attrs==19.1.0,coverage==4.5.4,importlib-metadata==0.19,more-itertools==7.2.0,packaging==19.1,pathlib2==2.3.4,pluggy==0.12.0,py==1.8.0,pyparsing==2.4.2,pytest==4.6.5,pytest-cov==2.7.1,scandir==1.10.0,six==1.12.0,wcwidth==0.1.7,zipp==0.5.2
  removing /tmp/tox-test/.tox/python/tmp
python start: run-test-pre
python run-test-pre: PYTHONHASHSEED='2830029615'
python finish: run-test-pre  after 0.00 seconds
python start: run-test
python run-test: commands[0] | python -c 'import os; print(os.environ.get('"'"'PATH'"'"')); print(os.environ.get('"'"'TEST'"'"'))'
setting PATH=/tmp/tox-test/.tox/python/bin:/grid/common/bin:/local/method/infrastructure/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/sbin:/grid/sfi/hpc/pbspro/v18.1.2p1/wrappers/bin:/grid/scp/hw/apps/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/local/method/bin


[21516] /tmp/tox-test$ /tmp/tox-test/.tox/python/bin/python -c 'import os; print(os.environ.get('"'"'PATH'"'"')); print(os.environ.get('"'"'TEST'"'"'))'
/tmp/tox-test/.tox/python/bin:/grid/common/bin:/local/method/infrastructure/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/sbin:/grid/sfi/hpc/pbspro/v18.1.2p1/wrappers/bin:/grid/scp/hw/apps/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/local/method/bin
test


python run-test: commands[1] | echo /path/to/binary
setting PATH=/tmp/tox-test/.tox/python/bin:/grid/common/bin:/local/method/infrastructure/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/bin:/grid/sfi/hpc/pbspro/v18.1.2p1/RHEL6/sbin:/grid/sfi/hpc/pbspro/v18.1.2p1/wrappers/bin:/grid/scp/hw/apps/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin:/local/method/bin


[21517] /tmp/tox-test$ /grid/common/bin/echo /path/to/binary
/path/to/binary


python finish: run-test  after 0.03 seconds
python start: run-test-post
python finish: run-test-post  after 0.00 seconds

The passenv = PATH doesn't change anything here.
In the documentation, I cannot find anything that would state that this is expected behavior.

@aklajnert aklajnert added the bug:normal label Sep 18, 2019
@asottile
Copy link
Member

@asottile asottile commented Sep 24, 2019

the way tox works is by creating a virtualenv, prefixing the PATH with that, and then invoking the commands -- does that help clarify things?

@aklajnert
Copy link
Author

@aklajnert aklajnert commented Sep 25, 2019

the way tox works is by creating a virtualenv, prefixing the PATH with that, and then invoking the commands -- does that help clarify things?

Yes - that helps understand why it is happening. This behavior can be however confusing and unexpected. I see a room for improvement here:

  • Warn that setting PATH will have no effect and don't set {env:PATH} since it won't be passed down.
  • Introduce something like appendenv and prependenv options, that will allow extending PATH and also other PATH-like environment variables.

Currently, the PATH can be only extended via setting it up before running tox or doing so within the command, which doesn't seem to be the right place.

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Apr 19, 2020

Feel free to open a PR against master and fix it. My available efforts at the moment are aimed at fixing this as part of #1394, but that probably will take a while (ETA September).

@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented May 2, 2020

Hello, this now has been released via https://pypi.org/project/tox/3.15.0/

heads up

@tox-dev tox-dev locked and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug:normal pr-available
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants