New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle interpreter directives with long lengths #794

Merged
merged 4 commits into from May 12, 2018

Conversation

Projects
None yet
5 participants
@jdknight
Contributor

jdknight commented Apr 24, 2018

handle interpreter directives with long lengths

When preparing virtual environments in a file container which has large length, the system might not be able to invoke shebang scripts which define interpreters beyond system limits (i.e. Linux has a limit of 128; BINPRM_BUF_SIZE [1]). This wrapper can be used to check if the executable is a script containing a shebang line [2]. If so, read the first line for the interpreter and check if length exceeds "common" limit. If the interpreter exceeds the limit, adjust the argument list to directly call the interpreter.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/include/uapi/linux/binfmts.h?h=linux-4.16.y#n19
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/fs/binfmt_script.c?h=linux-4.16.y#n24


Recently had an issue attempting to run tox in a Jenkins Multibranch Pipeline where I received the following error:

$ tox
GLOB sdist-make: /var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/setup.py
py27 create: /var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/.tox/py27
py27 installdeps: -r/var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/requirements.txt
ERROR: invocation failed (errno 2), args: ['/var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/.tox/py27/bin/pip', 'install', '-r/var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/requirements.txt'], cwd: /var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q
Traceback (most recent call last):
  File "/bin/tox", line 11, in <module>
    sys.exit(run_main())
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 40, in run_main
    main(args)
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 46, in main
    retcode = Session(config).runcommand()
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 415, in runcommand
    return self.subcommand_test()
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 599, in subcommand_test
    if self.setupenv(venv):
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 491, in setupenv
    status = venv.update(action=action)
  File "/usr/lib/python2.7/site-packages/tox/venv.py", line 171, in update
    self.hook.tox_testenv_install_deps(action=action, venv=self)
  File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 617, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 222, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python2.7/site-packages/pluggy/__init__.py", line 216, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/usr/lib/python2.7/site-packages/pluggy/callers.py", line 201, in _multicall
    return outcome.get_result()
  File "/usr/lib/python2.7/site-packages/pluggy/callers.py", line 77, in get_result
    _reraise(*ex)  # noqa
  File "/usr/lib/python2.7/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python2.7/site-packages/tox/venv.py", line 452, in tox_testenv_install_deps
    venv._install(deps, action=action)
  File "/usr/lib/python2.7/site-packages/tox/venv.py", line 331, in _install
    action=action)
  File "/usr/lib/python2.7/site-packages/tox/venv.py", line 303, in run_install_command
    action=action, redirect=self.session.report.verbosity < 2)
  File "/usr/lib/python2.7/site-packages/tox/venv.py", line 409, in _pcall
    redirect=redirect, ignore_ret=ignore_ret)
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 150, in popen
    stdout=stdout, stderr=subprocess.STDOUT)
  File "/usr/lib/python2.7/site-packages/tox/session.py", line 243, in _popen
    stdout=stdout, stderr=stderr, env=env)
  File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

My tox version:

$ tox --version
3.0.0 

After some investigation, it appears to be an issue with the path length generated by Jenkins and the interpreter strings of scripts tox needs to invoke. This issue has been also discussed in #66 and #649 (and related pypa/pip#1773, pypa/virtualenv#596). This proposed change is an attempt to help provide a workaround for tox user's in long-path environments. After the above change, I was able to invoke a tox-support project under a long-path environment:

$ tox
GLOB sdist-make: /var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/setup.py
py27 inst-nodeps: /var/lib/jenkins/workspace/[...]-py_initial-devwork-RC5P2SZZC5BWPRXRA37BMA5XOH53QUYKJ5Z2625THIQZFTY2FL6Q/.tox/dist/[...]-0.1.0.dev0.zip
py27 installed: enum34==1.1.6,[...]==0.1.0.dev0
py27 runtests: PYTHONHASHSEED='931962048'
...

(note: I don't know much about tox's internals, so this could be completely wrong; if workarounds like this are not desired, I have no problem in dropping this request)

@codecov

This comment has been minimized.

codecov bot commented Apr 24, 2018

Codecov Report

Merging #794 into master will decrease coverage by <1%.
The diff coverage is 77%.

@@          Coverage Diff           @@
##           master   #794    +/-   ##
======================================
- Coverage      95%    95%   -<1%     
======================================
  Files          11     11            
  Lines        2307   2320    +13     
======================================
+ Hits         2202   2212    +10     
- Misses        105    108     +3
@obestwalter

This comment has been minimized.

Member

obestwalter commented Apr 24, 2018

Hi @jdknight, thanks for having a go at this. I like the idea to only try this, when it would otherwise fail anyway, so this might be a good direction to go.

tox/venv.py Outdated
with open(args[0], 'rb') as f:
if f.read(1) == '#' and f.read(1) == '!':
interp = f.readline(1024).rstrip()
if len(interp) > 125:

This comment has been minimized.

@obestwalter

obestwalter Apr 24, 2018

Member

Why is it not possible to take the actual value of BINPRM_BUF_SIZE into account here?

This comment has been minimized.

@jdknight

jdknight Apr 25, 2018

Contributor

Truthfully, it was a guess to what the actual total interpret length could be 😅. I suspected that BINPRM_BUF_SIZE's value (i.e. 128) couldn't be used directly since I didn't know (or explicitly check) to whether or not the prefix #! and other conditions would be part of the "real limit". But yes, I would agree just throwing it this magic number here isn't really helpful for maintenance reasons.

I re-checked Linux's binfmt_script.c and (I believe) confirmed that a value of 125 should be the value. The define BINPRM_BUF_SIZE (128) with an enforced null-terminating character gives us 127 characters. The prefix #! does contribute to the maximum length, so we have a maximum of 125 characters for our interpreter string (at least, on Linux; 1). Instead of just having a value 127, we could do something like (a note why MAXINTERP's value exist in the comment):

MAXINTERP = 125
if len(interp) > MAXINTERP:

On a related note, it may be ideal to also change args = [interp] + args to be args = [interp.lstrip()] + args. This should trim leading whitespace characters from the interpreter before execution (which I assume is a good thing).

This comment has been minimized.

@obestwalter

obestwalter Apr 25, 2018

Member

The problem is that AFAIK different distros patch this length differently, so we would need to be able to know from within a running system. Another (ugly) approach would be to try it and react to that specific problem to try again with the adjusted approach ...

This comment has been minimized.

@jdknight

jdknight Apr 25, 2018

Contributor

Agreed that handling interpreter limits for various distributions is a problem. The lazy'er approach would be to just define the common maximum interpret length and allow the user to override the limit (through an environment variable?) to help force this workaround in a more limited-length environment.

The "ugly" approach could work. Execution the command as expected and if it fails with a ENOENT (and is !win32 and is a detected shebang script), try it again with the wrapped call (I'd worry if an executable actually returns a value ENOENT for some operating case; but what are the chances there...).

Another approach would be just to detect is the to be executed script is a shebang script and always wrap it.


The code interp = f.readline(1024).rstrip() probably needs more work as well:

  • In the event that we don't care about the interpreter's length, we could drop rstrip.
  • The size 1024 may be too small. In FreeBSD, MAXINTERP is set to PATH_MAX (2048).
  • The directive should probably be split on whitespaces (to handle scenarios where something like #!/usr/bin/env python set).
@gst

gst approved these changes Apr 27, 2018

I confirm it solves the problem in my case

@gst

This comment has been minimized.

Contributor

gst commented Apr 27, 2018

I guess adding some test(s) on this and it would be merge ready ?
there could be an eventual opt-in or opt-out flag as it's more a workaround IMO but making the job, and if it can be quickly integrated that would be awesome :)

@obestwalter

This comment has been minimized.

Member

obestwalter commented Apr 27, 2018

Hi @gst thanks for testing on your setup. Which system did you test on?

Regarding merging this: the minimum we need are some tests and a file in the changelog folder explaining the change.

I am still not sure if we should go for the retry option instead, but we would have to be able to make sure that we only retry if we can be sure that the problem was due to the shebang length.

We could merge then and test it out in an rc hoping we catch any problems. It is very hard to say if a hack like that might cause other problems down the road, so it would be good if more eyeballs would be involved.

@gst

This comment has been minimized.

Contributor

gst commented Apr 27, 2018

Ubuntu 16.04(.3). Python 2.7. Under a Jenkins instance (but not required).

I might have a look at adding some test for this (even if I don't know tox internals at all for now).

I'd make this feature workaround enabled by default but with an opt-out flag on my side.

@jdknight

This comment has been minimized.

Contributor

jdknight commented Apr 28, 2018

The following is another pass based off comments made in the active pull request. Changes made and reason for the changes include:

  • This workaround is disabled* by default and can be enabled on supported systems with the TOX_LIMITED_SHEBANG environment variable set (e.x. export TOX_LIMITED_SHEBANG=1).
  • No longer rely on an assume interpreter maximum length on a system. If an interpreter target shell script is detected, this workaround will always grab the interpreter to use and invoke the script explicitly (i.e. no longer relying on platform's shebang implementation).
  • Support the optional single argument supported by shebang interpreter definitions.
  • Increase the size of the parsed line to prevent issues on platforms which support an even larger interpreter length (e.x. FreeBSD supports up to 2048).
  • Adding unit testing for the prepend interpreter method (suggested by @obestwalter; which could probably be expanded on).

* I've (reluctantly) adjusted this workaround to be disabled by default (counter to what @gst has suggested). The reason is primarily due to the existing unit tests which have a certain expectations on the _pcall implementation. Changes could be made to these unit tests to support the workaround enabled by default, but I didn't want to attempt something like that without a second opinion. I also noticed that with the workaround enabled, the use of prepended interpreters can trigger warning reports about non-whitelisted externals. I've only seen a subset of this implementation, so there may be other implications that I haven't considered yet.

@obestwalter

This comment has been minimized.

Member

obestwalter commented Apr 28, 2018

I actually think that it is a very good idea to have this workaround disabled by default. We can definitely run with this, after we have the spring cleaning out of the way that is currently going on.

@gst

This comment has been minimized.

Contributor

gst commented Apr 28, 2018

finally agree it's safer to keep it disabled by default. 👍

@onlymellb

This comment has been minimized.

onlymellb commented May 9, 2018

This PR works for my case.

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented May 9, 2018

@jdknight please take time when you have to resolve conflicts and I'll take the time to review and merge 😄

venv: prepend interpreter directive to argument list
When preparing virtual environments in a file container which has
large length, the system might not be able to invoke shebang scripts
which define interpreters beyond system limits (e.x. Linux as a
limit of 128; BINPRM_BUF_SIZE [1]). This method can be used to check
if the executable is a script containing a shebang line [2]. If so,
extract the interpreter (and possible optional argument) and prepend
the values to the provided argument list. tox will only attempt to
read an interpreter directive of a maximum size of 2048 bytes to
limit excessive reading and support UNIX systems which may support a
longer interpret length.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/include/uapi/linux/binfmts.h?h=linux-4.16.y#n19
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/fs/binfmt_script.c?h=linux-4.16.y#n24

Signed-off-by: James Knight <james.d.knight@live.com>
@gaborbernat

gaborbernat requested changes May 10, 2018 edited

@jdknight there's one more thing missing, please add a changelog entry as in https://github.com/tox-dev/tox/blob/master/changelog/798.feature.rst

Also consider either parametrizing the test case, or just moving into multiple independent tests.

@jdknight

This comment has been minimized.

Contributor

jdknight commented May 10, 2018

@gaborbernat, sure; I'll append another commit with a changelog entry (~8-10PM EST).

As for your second comment, are you looking for something like this instead:

@pytest.mark.skipif("sys.platform == 'win32'")
def test_tox_testenv_interpret_shebang_empty_instance(tmpdir):
    testfile = tmpdir.join('test_shebang.py')
    base_args = [str(testfile), 'arg1', 'arg2', 'arg3']

    testfile.write('')
    args = prepend_shebang_interpreter(base_args)
    assert args == base_args

@pytest.mark.skipif("sys.platform == 'win32'")
def test_tox_testenv_interpret_shebang_empty_interp(tmpdir):
    testfile = tmpdir.join('test_shebang.py')
    base_args = [str(testfile), 'arg1', 'arg2', 'arg3']

    testfile.write('#!')
    args = prepend_shebang_interpreter(base_args)
    assert args == base_args

@pytest.mark.skipif("sys.platform == 'win32'")
def test_tox_testenv_interpret_shebang_empty_interpws(tmpdir):
    testfile = tmpdir.join('test_shebang.py')
    base_args = [str(testfile), 'arg1', 'arg2', 'arg3']

    testfile.write('#!    \n')
    args = prepend_shebang_interpreter(base_args)
    assert args == base_args
...
@gaborbernat

This comment has been minimized.

Member

gaborbernat commented May 10, 2018

@jdknight yes; tests which are independent should be I think separate, so we can see failures separately, and gives better options to parallel run it

tests: validate shebang interpreter prepending
Providing a series of test assertions to validate the recently added
method `prepend_shebang_interpreter`. Ensures shell scripts with various
interpreter directives (whitespaces, single argument, invalid entries)
are appended or ignored respectfully.

Signed-off-by: James Knight <james.d.knight@live.com>
@gaborbernat

This comment has been minimized.

Member

gaborbernat commented May 11, 2018

Given this thing is optional can you amend the description how to enable it? Maybe even in the documentation beside the changelog 😀

@jdknight

This comment has been minimized.

Contributor

jdknight commented May 11, 2018

@gaborbernat, sure. I did not know what the expected/desired content in a changelog entry was. I just assumed, based on what I saw in existing file entries, is to just include a simple line description trailed by ownership. After now examining the content I see in CHANGELOG.rst, I'll tweak the changelog entry to be more descriptive.

changelog: track pr 794
Track change which supports handling interpreter directives in
environments with long path lengths [1].

[1]: e913dd3

Signed-off-by: James Knight <james.d.knight@live.com>
@gaborbernat

This comment has been minimized.

Member

gaborbernat commented May 11, 2018

@jdknight really sorry to be a hassle about this, but just one final thing add a note about this to our config sections end https://github.com/tox-dev/tox/blob/master/doc/config.rst

@jdknight

This comment has been minimized.

Contributor

jdknight commented May 12, 2018

Before I try to submit something for review, any objections for me to add level-two title (a child of tox configuration specification) at the bottom (before Other Rules and notes) called "advanced settings"? Under this section I can make a subsection describing this workaround.

@gaborbernat

This comment has been minimized.

Member

gaborbernat commented May 12, 2018

Yeah I think that's a good approach.

doc: add workaround for limited shebang environments
Adding documentation to tox's configuration document describing how
to take advantage of the `TOX_LIMITED_SHEBANG` environment variable
to bypass system-defined shebang interpreter directive limits.

Signed-off-by: James Knight <james.d.knight@live.com>

@gaborbernat gaborbernat merged commit 65c5ed3 into tox-dev:master May 12, 2018

1 of 2 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jdknight

This comment has been minimized.

Contributor

jdknight commented May 12, 2018

Thanks everyone.

@jdknight jdknight deleted the jdknight:shebang-length-limit-wrapper branch May 12, 2018

sfdye added a commit to pydanny/cookiecutter-django that referenced this pull request Jul 10, 2018

Update tox to 3.1.1 (#1715)
This PR updates [tox](https://pypi.org/project/tox) from **3.0.0** to **3.1.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.1.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- Add ``ignore_basepython_conflict``, which determines whether conflicting
  ``basepython`` settings for environments containing default factors, such as
  ``py27`` or ``django18-py35``, should be ignored or result in warnings. This
  was a common source of misconfiguration and is rarely, if ever, desirable from
  a user perspective - by :user:`stephenfin` (`477 &lt;https://github.com/tox-dev/tox/issues/477&gt;`_)
- Fix bug with incorrectly defactorized dependencies (deps passed to pip were not de-factorized) - by :user:`bartsanchez` (`706 &lt;https://github.com/tox-dev/tox/issues/706&gt;`_)


Features
^^^^^^^^

- Add support for multiple PyPy versions using default factors. This allows you
  to use, for example, ``pypy27`` knowing that the correct intepreter will be
  used by default - by :user:`stephenfin` (`19 &lt;https://github.com/tox-dev/tox/issues/19&gt;`_)
- Add support to explicitly invoke interpreter directives for environments with
  long path lengths. In the event that ``tox`` cannot invoke scripts with a
  system-limited shebang (e.x. a Linux host running a Jenkins Pipeline), a user
  can set the environment variable ``TOX_LIMITED_SHEBANG`` to workaround the
  system&#39;s limitation (e.x. ``export TOX_LIMITED_SHEBANG=1``) - by :user:`jdknight` (`794 &lt;https://github.com/tox-dev/tox/issues/794&gt;`_)
- introduce a constants module to be used internally and as experimental API - by :user:`obestwalter` (`798 &lt;https://github.com/tox-dev/tox/issues/798&gt;`_)
- Make ``py2`` and ``py3`` aliases also resolve via ``py`` on windows by :user:`asottile`. This enables the following things:
  ``tox -e py2`` and ``tox -e py3`` work on windows (they already work on posix); and setting ``basepython=python2`` or ``basepython=python3`` now works on windows. (`856 &lt;https://github.com/tox-dev/tox/issues/856&gt;`_)
- Replace the internal version parsing logic from the not well tested `PEP-386 &lt;https://www.python.org/dev/peps/pep-0386/&gt;`_ parser for the more general `PEP-440 &lt;https://www.python.org/dev/peps/pep-0440/&gt;`_. `packaging &gt;= 17.1 &lt;https://pypi.org/project/packaging/&gt;`_ is now an install dependency by :user:`gaborbernat`. (`860 &lt;https://github.com/tox-dev/tox/issues/860&gt;`_)


Documentation
^^^^^^^^^^^^^

- extend the plugin documentation and make lot of small fixes and improvements - by :user:`obestwalter` (`797 &lt;https://github.com/tox-dev/tox/issues/797&gt;`_)
- tidy up tests - remove unused fixtures, update old cinstructs, etc. - by :user:`obestwalter` (`799 &lt;https://github.com/tox-dev/tox/issues/799&gt;`_)
- Various improvements to documentation: open browser once documentation generation is done, show Github/Travis info on documentation page, remove duplicate header for changelog, generate unreleased news as DRAFT on top of changelog, make the changelog page more compact and readable (width up to 1280px) by :user:`gaborbernat` (`859 &lt;https://github.com/tox-dev/tox/issues/859&gt;`_)


Miscellaneous
^^^^^^^^^^^^^

- filter out unwanted files in package - by :user:`obestwalter` (`754 &lt;https://github.com/tox-dev/tox/issues/754&gt;`_)
- make the already existing implicit API explicit - by :user:`obestwalter` (`800 &lt;https://github.com/tox-dev/tox/issues/800&gt;`_)
- improve tox quickstart and corresponding tests - by :user:`obestwalter` (`801 &lt;https://github.com/tox-dev/tox/issues/801&gt;`_)
- tweak codecov settings via .codecov.yml - by :user:`obestwalter` (`802 &lt;https://github.com/tox-dev/tox/issues/802&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/tox
  - Changelog: https://pyup.io/changelogs/tox/
  - Docs: https://tox.readthedocs.org/
</details>

sloria added a commit to sloria/doitlive that referenced this pull request Jul 15, 2018

Update tox to 3.1.2 (#56)
This PR updates [tox](https://pypi.org/project/tox) from **3.0.0** to **3.1.2**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.1.1
   ```
   ------------------

Bugfixes
^^^^^^^^

- PyPi documentation for ``3.1.0`` is broken. Added test to check for this, and
  fix it by :user:`gaborbernat`. (`879
  &lt;https://github.com/tox-dev/tox/issues/879&gt;`_)
   ```
   
  
  
   ### 3.1.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- Add ``ignore_basepython_conflict``, which determines whether conflicting
  ``basepython`` settings for environments containing default factors, such as
  ``py27`` or ``django18-py35``, should be ignored or result in warnings. This
  was a common source of misconfiguration and is rarely, if ever, desirable from
  a user perspective - by :user:`stephenfin` (`477 &lt;https://github.com/tox-dev/tox/issues/477&gt;`_)
- Fix bug with incorrectly defactorized dependencies (deps passed to pip were not de-factorized) - by :user:`bartsanchez` (`706 &lt;https://github.com/tox-dev/tox/issues/706&gt;`_)


Features
^^^^^^^^

- Add support for multiple PyPy versions using default factors. This allows you
  to use, for example, ``pypy27`` knowing that the correct intepreter will be
  used by default - by :user:`stephenfin` (`19 &lt;https://github.com/tox-dev/tox/issues/19&gt;`_)
- Add support to explicitly invoke interpreter directives for environments with
  long path lengths. In the event that ``tox`` cannot invoke scripts with a
  system-limited shebang (e.x. a Linux host running a Jenkins Pipeline), a user
  can set the environment variable ``TOX_LIMITED_SHEBANG`` to workaround the
  system&#39;s limitation (e.x. ``export TOX_LIMITED_SHEBANG=1``) - by :user:`jdknight` (`794 &lt;https://github.com/tox-dev/tox/issues/794&gt;`_)
- introduce a constants module to be used internally and as experimental API - by :user:`obestwalter` (`798 &lt;https://github.com/tox-dev/tox/issues/798&gt;`_)
- Make ``py2`` and ``py3`` aliases also resolve via ``py`` on windows by :user:`asottile`. This enables the following things:
  ``tox -e py2`` and ``tox -e py3`` work on windows (they already work on posix); and setting ``basepython=python2`` or ``basepython=python3`` now works on windows. (`856 &lt;https://github.com/tox-dev/tox/issues/856&gt;`_)
- Replace the internal version parsing logic from the not well tested `PEP-386 &lt;https://www.python.org/dev/peps/pep-0386/&gt;`_ parser for the more general `PEP-440 &lt;https://www.python.org/dev/peps/pep-0440/&gt;`_. `packaging &gt;= 17.1 &lt;https://pypi.org/project/packaging/&gt;`_ is now an install dependency by :user:`gaborbernat`. (`860 &lt;https://github.com/tox-dev/tox/issues/860&gt;`_)


Documentation
^^^^^^^^^^^^^

- extend the plugin documentation and make lot of small fixes and improvements - by :user:`obestwalter` (`797 &lt;https://github.com/tox-dev/tox/issues/797&gt;`_)
- tidy up tests - remove unused fixtures, update old cinstructs, etc. - by :user:`obestwalter` (`799 &lt;https://github.com/tox-dev/tox/issues/799&gt;`_)
- Various improvements to documentation: open browser once documentation generation is done, show Github/Travis info on documentation page, remove duplicate header for changelog, generate unreleased news as DRAFT on top of changelog, make the changelog page more compact and readable (width up to 1280px) by :user:`gaborbernat` (`859 &lt;https://github.com/tox-dev/tox/issues/859&gt;`_)


Miscellaneous
^^^^^^^^^^^^^

- filter out unwanted files in package - by :user:`obestwalter` (`754 &lt;https://github.com/tox-dev/tox/issues/754&gt;`_)
- make the already existing implicit API explicit - by :user:`obestwalter` (`800 &lt;https://github.com/tox-dev/tox/issues/800&gt;`_)
- improve tox quickstart and corresponding tests - by :user:`obestwalter` (`801 &lt;https://github.com/tox-dev/tox/issues/801&gt;`_)
- tweak codecov settings via .codecov.yml - by :user:`obestwalter` (`802 &lt;https://github.com/tox-dev/tox/issues/802&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/tox
  - Changelog: https://pyup.io/changelogs/tox/
  - Docs: https://tox.readthedocs.org/
</details>

jdknight added a commit to jdknight/tox that referenced this pull request Jul 24, 2018

doc: mention additional workarounds for jenkins shebang limits
tox provides an alternative to workaround system limits imposed on
shebang lines (`TOX_LIMITED_SHEBANG`, see tox-dev#794). Updating the Jenkins
example documentation to reflect this.

Signed-off-by: James Knight <james.d.knight@live.com>

gaborbernat added a commit that referenced this pull request Jul 24, 2018

doc: mention additional workarounds for jenkins shebang limits (#923)
tox provides an alternative to workaround system limits imposed on shebang lines (`TOX_LIMITED_SHEBANG`, see #794). Updating the Jenkins example documentation to reflect this.

DmytroLitvinov added a commit to DmytroLitvinov/kuna that referenced this pull request Aug 6, 2018

Scheduled weekly dependency update for week 31 (#19)



### Update [pip](https://pypi.org/project/pip) from **10.0.1** to **18.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 18.0
   ```
   =================

Process
-------

- Switch to a Calendar based versioning scheme.
- Formally document our deprecation process as a minimum of 6 months of deprecation
  warnings.
- Adopt and document NEWS fragment writing style.
- Switch to releasing a new, non bug fix version of pip every 3 months.

Deprecations and Removals
-------------------------

- Remove the legacy format from pip list. (3651, 3654)
- Dropped support for Python 3.3. (3796)
- Remove support for cleaning up egg fragment postfixes. (4174)
- Remove the shim for the old get-pip.py location. (5520)

  For the past 2 years, it&#39;s only been redirecting users to use the newer
  https://bootstrap.pypa.io/get-pip.py location.

Features
--------

- Introduce a new --prefer-binary flag, to prefer older wheels over newer source packages. (3785)
- Improve autocompletion function on file name completion after options
  which have ``&lt;file&gt;``, ``&lt;dir&gt;`` or ``&lt;path&gt;`` as metavar. (4842, 5125)
- Add support for installing PEP 518 build dependencies from source. (5229)
- Improve status message when upgrade is skipped due to only-if-needed strategy. (5319)

Bug Fixes
---------

- Update pip&#39;s self-check logic to not use a virtualenv specific file and honor cache-dir. (3905)
- Remove compiled pyo files for wheel packages. (4471)
- Speed up printing of newly installed package versions. (5127)
- Restrict install time dependency warnings to directly-dependant packages. (5196, 5457)

  Warning about the entire package set has resulted in users getting confused as
  to why pip is printing these warnings.
- Improve handling of PEP 518 build requirements: support environment markers and extras. (5230, 5265)
- Remove username/password from log message when using index with basic auth. (5249)
- Remove trailing os.sep from PATH directories to avoid false negatives. (5293)
- Fix &quot;pip wheel pip&quot; being blocked by the &quot;don&#39;t use pip to modify itself&quot; check. (5311, 5312)
- Disable pip&#39;s version check (and upgrade message) when installed by a different package manager. (5346)

  This works better with Linux distributions where pip&#39;s upgrade message may
  result in users running pip in a manner that modifies files that should be
  managed by the OS&#39;s package manager.
- Check for file existence and unlink first when clobbering existing files during a wheel install. (5366)
- Improve error message to be more specific when no files are found as listed in as listed in PKG-INFO. (5381)
- Always read ``pyproject.toml`` as UTF-8. This fixes Unicode handling on Windows and Python 2. (5482)
- Fix a crash that occurs when PATH not set, while generating script location warning. (5558)
- Disallow packages with ``pyproject.toml`` files that have an empty build-system table. (5627)

Vendored Libraries
------------------

- Update CacheControl to 0.12.5.
- Update certifi to 2018.4.16.
- Update distro to 1.3.0.
- Update idna to 2.7.
- Update ipaddress to 1.0.22.
- Update pkg_resources to 39.2.0 (via setuptools).
- Update progress to 1.4.
- Update pytoml to 0.1.16.
- Update requests to 2.19.1.
- Update urllib3 to 1.23.

Improved Documentation
----------------------

- Document how to use pip with a proxy server. (512, 5574)
- Document that the output of pip show is in RFC-compliant mail header format. (5261)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pip
  - Changelog: https://pyup.io/changelogs/pip/
  - Homepage: https://pip.pypa.io/
</details>





### Update [tox](https://pypi.org/project/tox) from **3.0.0** to **3.1.3**.


<details>
  <summary>Changelog</summary>
  
  
   ### 3.1.2
   ```
   ------------------

Bugfixes
^^^^^^^^

- Revert &quot;Fix bug with incorrectly defactorized dependencies (`772 &lt;https://github.com/tox-dev/tox/issues/772&gt;`_)&quot; due to a regression (`(799) &lt;https://github.com/tox-dev/tox/issues/899&gt;`_) - by :user:`obestwalter`
   ```
   
  
  
   ### 3.1.1
   ```
   ------------------

Bugfixes
^^^^^^^^

- PyPi documentation for ``3.1.0`` is broken. Added test to check for this, and
  fix it by :user:`gaborbernat`. (`879
  &lt;https://github.com/tox-dev/tox/issues/879&gt;`_)
   ```
   
  
  
   ### 3.1.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- Add ``ignore_basepython_conflict``, which determines whether conflicting
  ``basepython`` settings for environments containing default factors, such as
  ``py27`` or ``django18-py35``, should be ignored or result in warnings. This
  was a common source of misconfiguration and is rarely, if ever, desirable from
  a user perspective - by :user:`stephenfin` (`477 &lt;https://github.com/tox-dev/tox/issues/477&gt;`_)
- Fix bug with incorrectly defactorized dependencies (deps passed to pip were not de-factorized) - by :user:`bartsanchez` (`706 &lt;https://github.com/tox-dev/tox/issues/706&gt;`_)


Features
^^^^^^^^

- Add support for multiple PyPy versions using default factors. This allows you
  to use, for example, ``pypy27`` knowing that the correct intepreter will be
  used by default - by :user:`stephenfin` (`19 &lt;https://github.com/tox-dev/tox/issues/19&gt;`_)
- Add support to explicitly invoke interpreter directives for environments with
  long path lengths. In the event that ``tox`` cannot invoke scripts with a
  system-limited shebang (e.x. a Linux host running a Jenkins Pipeline), a user
  can set the environment variable ``TOX_LIMITED_SHEBANG`` to workaround the
  system&#39;s limitation (e.x. ``export TOX_LIMITED_SHEBANG=1``) - by :user:`jdknight` (`794 &lt;https://github.com/tox-dev/tox/issues/794&gt;`_)
- introduce a constants module to be used internally and as experimental API - by :user:`obestwalter` (`798 &lt;https://github.com/tox-dev/tox/issues/798&gt;`_)
- Make ``py2`` and ``py3`` aliases also resolve via ``py`` on windows by :user:`asottile`. This enables the following things:
  ``tox -e py2`` and ``tox -e py3`` work on windows (they already work on posix); and setting ``basepython=python2`` or ``basepython=python3`` now works on windows. (`856 &lt;https://github.com/tox-dev/tox/issues/856&gt;`_)
- Replace the internal version parsing logic from the not well tested `PEP-386 &lt;https://www.python.org/dev/peps/pep-0386/&gt;`_ parser for the more general `PEP-440 &lt;https://www.python.org/dev/peps/pep-0440/&gt;`_. `packaging &gt;= 17.1 &lt;https://pypi.org/project/packaging/&gt;`_ is now an install dependency by :user:`gaborbernat`. (`860 &lt;https://github.com/tox-dev/tox/issues/860&gt;`_)


Documentation
^^^^^^^^^^^^^

- extend the plugin documentation and make lot of small fixes and improvements - by :user:`obestwalter` (`797 &lt;https://github.com/tox-dev/tox/issues/797&gt;`_)
- tidy up tests - remove unused fixtures, update old cinstructs, etc. - by :user:`obestwalter` (`799 &lt;https://github.com/tox-dev/tox/issues/799&gt;`_)
- Various improvements to documentation: open browser once documentation generation is done, show Github/Travis info on documentation page, remove duplicate header for changelog, generate unreleased news as DRAFT on top of changelog, make the changelog page more compact and readable (width up to 1280px) by :user:`gaborbernat` (`859 &lt;https://github.com/tox-dev/tox/issues/859&gt;`_)


Miscellaneous
^^^^^^^^^^^^^

- filter out unwanted files in package - by :user:`obestwalter` (`754 &lt;https://github.com/tox-dev/tox/issues/754&gt;`_)
- make the already existing implicit API explicit - by :user:`obestwalter` (`800 &lt;https://github.com/tox-dev/tox/issues/800&gt;`_)
- improve tox quickstart and corresponding tests - by :user:`obestwalter` (`801 &lt;https://github.com/tox-dev/tox/issues/801&gt;`_)
- tweak codecov settings via .codecov.yml - by :user:`obestwalter` (`802 &lt;https://github.com/tox-dev/tox/issues/802&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/tox
  - Changelog: https://pyup.io/changelogs/tox/
  - Docs: https://tox.readthedocs.org/
</details>





### Update [Sphinx](https://pypi.org/project/Sphinx) from **1.7.5** to **1.7.6**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.7.6
   ```
   ==============================

Dependencies
------------

Incompatible changes
--------------------

Deprecated
----------

Features added
--------------

Bugs fixed
----------

* 5037: LaTeX ``\sphinxupquote{}`` breaks in Russian
* sphinx.testing uses deprecated pytest API; ``Node.get_marker(name)``
* 5016: crashed when recommonmark.AutoStrictify is enabled
* 5022: latex: crashed with docutils package provided by Debian/Ubuntu
* 5009: latex: a label for table is vanished if table does not have a caption
* 5048: crashed with numbered toctree
* 2410: C, render empty argument lists for macros.
* C++, fix lookup of full template specializations with no template arguments.
* 4667: C++, fix assertion on missing references in global scope when using
  intersphinx. Thanks to Alan M. Carroll.
* 5019: autodoc: crashed by Form Feed Character
* 5032: autodoc: loses the first staticmethod parameter for old styled classes
* 5036: quickstart: Typing Ctrl-U clears the whole of line
* 5066: html: &quot;relations&quot; sidebar is not shown by default
* 5091: latex: curly braces in index entries are not handled correctly
* 5070: epub: Wrong internal href fragment links
* 5104: apidoc: Interface of ``sphinx.apidoc:main()`` has changed
* 5076: napoleon raises RuntimeError with python 3.7
* 5125: sphinx-build: Interface of ``sphinx:main()`` has changed
* sphinx-build: ``sphinx.cmd.build.main()`` refers ``sys.argv`` instead of given
  argument
* 5146: autosummary: warning is emitted when the first line of docstring ends
  with literal notation
* autosummary: warnings of autosummary indicates wrong location (refs: 5146)
* 5143: autodoc: crashed on inspecting dict like object which does not support
  sorting

Testing
--------
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/sphinx
  - Changelog: https://pyup.io/changelogs/sphinx/
  - Homepage: http://sphinx-doc.org/
</details>





### Update [cryptography](https://pypi.org/project/cryptography) from **2.2.2** to **2.3**.


<details>
  <summary>Changelog</summary>
  
  
   ### 2.3
   ```
   ~~~~~~~~~~~~~~~~

* **SECURITY ISSUE:**
  :meth:`~cryptography.hazmat.primitives.ciphers.AEADDecryptionContext.finalize_with_tag`
  allowed tag truncation by default which can allow tag forgery in some cases.
  The method now enforces the ``min_tag_length`` provided to the
  :class:`~cryptography.hazmat.primitives.ciphers.modes.GCM` constructor.
  *CVE-2018-10903*
* Added support for Python 3.7.
* Added :meth:`~cryptography.fernet.Fernet.extract_timestamp` to get the
  authenticated timestamp of a :doc:`Fernet &lt;/fernet&gt;` token.
* Support for Python 2.7.x without ``hmac.compare_digest`` has been deprecated.
  We will require Python 2.7.7 or higher (or 2.7.6 on Ubuntu) in the next
  ``cryptography`` release.
* Fixed multiple issues preventing ``cryptography`` from compiling against
  LibreSSL 2.7.x.
* Added
  :class:`~cryptography.x509.CertificateRevocationList.get_revoked_certificate_by_serial_number`
  for quick serial number searches in CRLs.
* The :class:`~cryptography.x509.RelativeDistinguishedName` class now
  preserves the order of attributes. Duplicate attributes now raise an error
  instead of silently discarding duplicates.
* :func:`~cryptography.hazmat.primitives.keywrap.aes_key_unwrap` and
  :func:`~cryptography.hazmat.primitives.keywrap.aes_key_unwrap_with_padding`
  now raise :class:`~cryptography.hazmat.primitives.keywrap.InvalidUnwrap` if
  the wrapped key is an invalid length, instead of ``ValueError``.

.. _v2-2-2:
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/cryptography
  - Changelog: https://pyup.io/changelogs/cryptography/
  - Repo: https://github.com/pyca/cryptography
</details>





### Update [PyYAML](https://pypi.org/project/PyYAML) from **3.12** to **3.13**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pyyaml
  - Homepage: http://pyyaml.org/wiki/PyYAML
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment