Parallel mode with -e ALL fails

[guahki]

Using -p together with -e ALL runs all environments in all subprocesses. Using -e <list,of,all,envs> on the other hand works as expected (each subprocess runs one environment).

Minimal example:

tox.ini

[tox]
envlist = py

[testenv]
deps = requests
skip_install = true
commands = python -V

[testenv:lint]
commands = python -c "print('Hello lint')"

[testenv:dev]
commands = python -c "print('Hello dev')"

Environment:

$ tox --version
3.24.2 imported from ~\.local\pipx\venvs\tox\lib\site-packages\tox\__init__.py
$ ~/.local/pipx/venvs/tox/Scripts/python -V
Python 3.9.6

Correct command:

$ tox -p -o -e py,lint,dev
py create: ~\Dev\tmp\test_tox\.tox\py
dev create: ~\Dev\tmp\test_tox\.tox\dev
lint create: ~\Dev\tmp\test_tox\.tox\lint
py installdeps: requests
dev installdeps: requests
lint installdeps: requests
py installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
py run-test-pre: PYTHONHASHSEED='937'
py run-test: commands[0] | python -V
dev installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
dev run-test-pre: PYTHONHASHSEED='633'
dev run-test: commands[0] | python -c 'print('"'"'Hello dev'"'"')'
lint installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
lint run-test-pre: PYTHONHASHSEED='783'
lint run-test: commands[0] | python -c 'print('"'"'Hello lint'"'"')'
Python 3.9.7
✔ OK py in 8.858 seconds
Hello dev
Hello lint
✔ OK dev in 8.929 seconds
✔ OK lint in 8.936 seconds
____________________________________________________________ summary ____________________________________________________________
  py: commands succeeded
  lint: commands succeeded
  dev: commands succeeded
  congratulations :)

Incorrect behaviour:

$ tox -p -o -e ALL
py installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
py run-test-pre: PYTHONHASHSEED='672'
py run-test: commands[0] | python -V
py installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
py run-test-pre: PYTHONHASHSEED='524'
py run-test: commands[0] | python -V
py installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
py run-test-pre: PYTHONHASHSEED='977'
py run-test: commands[0] | python -V
Python 3.9.7
Python 3.9.7
Python 3.9.7
lint installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
lint installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
lint run-test-pre: PYTHONHASHSEED='524'
lint run-test: commands[0] | python -c 'print('"'"'Hello lint'"'"')'
lint run-test-pre: PYTHONHASHSEED='672'
lint run-test: commands[0] | python -c 'print('"'"'Hello lint'"'"')'
lint installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
lint run-test-pre: PYTHONHASHSEED='977'
lint run-test: commands[0] | python -c 'print('"'"'Hello lint'"'"')'
Hello lint
Hello lint
Hello lint
dev installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
dev run-test-pre: PYTHONHASHSEED='524'
dev run-test: commands[0] | python -c 'print('"'"'Hello dev'"'"')'
dev installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
dev run-test-pre: PYTHONHASHSEED='672'
dev run-test: commands[0] | python -c 'print('"'"'Hello dev'"'"')'
dev installed: certifi==2021.5.30,charset-normalizer==2.0.4,idna==3.2,requests==2.26.0,urllib3==1.26.6
dev run-test-pre: PYTHONHASHSEED='977'
dev run-test: commands[0] | python -c 'print('"'"'Hello dev'"'"')'
Hello dev
Hello dev
Hello dev
✔ OK dev in 5.777 seconds
✔ OK py in 5.802 seconds
✔ OK lint in 5.812 seconds
____________________________________________________________ summary ____________________________________________________________
  py: commands succeeded
  lint: commands succeeded
  dev: commands succeeded
  congratulations :)

Please note, that in the incorrect case, all three echos happen three times.

The issues comes from

tox/src/tox/config/__init__.py

Line 1501 in 156d496

if (from_option and "ALL" in from_option) or (

, which causes

tox/src/tox/config/__init__.py

Line 1507 in 156d496

(os.environ.get(PARALLEL_ENV_VAR_KEY_PRIVATE), True),

to not correctly restrict the env_list to the environment given by os.environ.get(PARALLEL_ENV_VAR_KEY_PRIVATE).

[guahki]

I further investigated the issue and corrected the bug-report above.

Adding env_list = [os.environ.get(PARALLEL_ENV_VAR_KEY_PRIVATE)] if os.environ.get(PARALLEL_ENV_VAR_KEY_PRIVATE) else [] before

tox/src/tox/config/__init__.py

Line 1504 in 156d496

all_envs = self._getallenvs(reader)

solves the issue. But I'm not sure, this is the correct solution.

Can somebody with more knowledge about _getenvdata in the config module please shed some light on how to fix this correctly?

[gaborbernat]

Released via https://tox.readthedocs.io/en/latest/changelog.html#v3-24-4-2021-09-16