New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factor with line continuations in commands does not work as expected #945

Open
wcooley opened this Issue Aug 15, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@wcooley

wcooley commented Aug 15, 2018

  • OS: Both MacOS and Linux
  • Python (tox's own venv): 3.6.4 & 3.6.5
  • Tox 3.1.2 and 3.2.1

Given a tox.ini like the following:

[tox]
envlist = py27,py3
skipsdist = True

[testenv]
commands =
    python -c 'from __future__ import print_function; import sys; print(sys.argv[1:])' \
        one \
        two \
        three

    py3: python -c 'print("This is Python 3-only")'

    py3: python -c 'import sys; print(sys.argv[1:])' \
            four \
            five \
            six

I expect that in the py27 env only the first python command runs and prints all three args. Instead what happens is that first command runs, the second is correctly skipped and the actual python line of the last is skipped but it attempts to run the command four five six:

py27 runtests: PYTHONHASHSEED='3628462677'
py27 runtests: commands[0] | python -c 'from __future__ import print_function; import sys; print(sys.argv[1:])' one two three
['one', 'two', 'three']
py27 runtests: commands[1] | four five six
ERROR: InvocationError for command could not find executable 'four'
py3 runtests: PYTHONHASHSEED='3628462677'
py3 runtests: commands[0] | python -c 'from __future__ import print_function; import sys; print(sys.argv[1:])' one two three
['one', 'two', 'three']
py3 runtests: commands[1] | python -c 'print("This is Python 3-only")'
This is Python 3-only
py3 runtests: commands[2] | python -c 'import sys; print(sys.argv[1:])' four five six
['four', 'five', 'six']
___________________________________ summary ____________________________________
ERROR:   py27: commands failed
  py3: commands succeeded

If four five six had indeed been a destructive command, this would be a most unpleasant surprise.

Curiously enough, I found nothing about line continuations in either the Tox docs or the (Py3) configparser docs; at least, I was unable to find anything relevant searching for "continuation", "newline", "break", "backslash", "linebreak" or "long". Eventually I found this in the documentation for commands=ARGLIST:

a command can have multiple lines if a line ends with the \ character in which case the subsequent line will be appended (and may contain another \ character …)

Further surprise: Prefixing all of the lines of the last command with py3 works -- it is both excluded from py27 and executed as one line in py3:

[testenv]
   ...
    py3: python -c 'import sys; print(sys.argv[1:])' \
    py3: four \
    py3: five \
    py3: six

Results in:

py27 runtests: PYTHONHASHSEED='2082075020'
py27 runtests: commands[0] | python -c 'from __future__ import print_function; import sys; print(sys.argv[1:])' one two three
['one', 'two', 'three']
py3 runtests: PYTHONHASHSEED='2082075020'
py3 runtests: commands[0] | python -c 'from __future__ import print_function; import sys; print(sys.argv[1:])' one two three
['one', 'two', 'three']
py3 runtests: commands[1] | python -c 'print("This is Python 3-only")'
This is Python 3-only
py3 runtests: commands[2] | python -c 'import sys; print(sys.argv[1:])' four five six
['four', 'five', 'six']
___________________________________ summary ____________________________________
  py27: commands succeeded
  py3: commands succeeded
  congratulations :)

So I am not sure if this is a documentation bug, in that the combination of factor with line continuations should be mentioned specifically, or if this is a feature bug/request, so that my
first tox.ini works as I expect it. I think the actual behavior is surprising but it is at least usable.

It would be good if the docs could include a searchable keyword in the documentation for "command=ARGLIST"; a backslash character cannot generally be searched for outside of grep.
Something like changing:

Each command is defined by one or more lines; a command can have multiple lines if a line ends with the \ character in which case the subsequent line will be appended (and may contain another \ character …).

To:

Each command is defined by one or more lines; a command can span multiple lines by ending the line with the backslash ("") character in which case the subsequent line will be appended (and may contain another \ character …), just as line continuations are implemented in Python itself.

And optionally add:

To split a command across multiple lines with a factor condition, all of the lines must be prefixed with the factor condition in addition to ending with the backslash character::

[testenv]
   ...
    py3: python -c 'import sys; print(sys.argv[1:])' \
    py3: four \
    py3: five \
    py3: six

I will attempt to put together a PR for the documentation in the near future but I am already quite busy right now.

@gaborbernat

This comment has been minimized.

Show comment
Hide comment
@gaborbernat

gaborbernat Sep 2, 2018

Member

Thanks @wcooley, let me know how that PR will go.

Member

gaborbernat commented Sep 2, 2018

Thanks @wcooley, let me know how that PR will go.

@gaborbernat gaborbernat added this to the 3.5 milestone Sep 18, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment