# Upgrading pip fails on Windows when install path is too long #3055

Closed
opened this Issue Aug 24, 2015 · 38 comments

Projects
None yet
9 participants
Contributor

### zooba commented Aug 24, 2015

 Received the following log on Microsoft/PTVS#782. Virtual environment is being created at 'C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env' Virtual environment was successfully created at 'C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env' ----- Installing 'pip' ----- You are using pip version 6.0.8, however version 7.1.2 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-7.1.2-py2.py3-none-any.whl#md5=5ff9fec0be479e4e36df467556deed4d Using cached pip-7.1.2-py2.py3-none-any.whl Installing collected packages: pip Found existing installation: pip 6.0.8 Uninstalling pip-6.0.8: Exception: Traceback (most recent call last): File "C:\Python34\lib\shutil.py", line 523, in move os.rename(src, real_dst) FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc' -> 'C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\basecommand.py", line 232, in main status = self.run(options, args) File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\commands\install.py", line 347, in run root=options.root_path, File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\req\req_set.py", line 543, in install requirement.uninstall(auto_confirm=True) File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\req\req_install.py", line 667, in uninstall paths_to_remove.remove(auto_confirm) File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\req\req_uninstall.py", line 126, in remove renames(path, new_path) File "C:\Users\TrevorSullivan\Source\Repos\PythonApplication11\PythonApplication11\env\lib\site-packages\pip\utils\__init__.py", line 316, in renames shutil.move(old, new) File "C:\Python34\lib\shutil.py", line 535, in move copy2(src, real_dst) File "C:\Python34\lib\shutil.py", line 245, in copy2 copyfile(src, dst, follow_symlinks=follow_symlinks) File "C:\Python34\lib\shutil.py", line 109, in copyfile with open(dst, 'wb') as fdst: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc' ----- Failed to install 'pip' ----- The problem seems to be that the entire install path is replicated beneath TEMP, which very quickly exceeds the maximum path length supported by Windows. I guess the aim is to be able to rollback a failed uninstall (which also fails here, and leaves corrupt state), but we may need an alternative to including the full path - maybe generate some sort of map file as well?
Contributor

### xavfernandez commented Sep 1, 2015

 Related to #2892

### RussBaz commented Feb 22, 2016

 I am still experiencing the same issue even though I am using pip 7.1.2 and python 3.5.1. Are there any updates available regarding this issue?

### brettcannon commented Aug 10, 2016 • edited

 Just ran into this under pip 8.1.1 and Python 3.5.2 on Windows 10.
Contributor

### GadgetSteve commented Oct 27, 2016 • edited

 I am wondering if a work around for Windows users might not be to check for paths that are likely to exceed the limit, (260 chars total), and using subst to mount a drive at a suitable point, i.e. identify a spare drive letter and mount that letter to as the TEMP - of course this would not truly fix the problem and deeply nested packages could still hit problems - that is down to Microsoft actually addressing the MAXPATH issue once and for all. If the installation succeeds then the temporary area could be deleted and the subst deleted. In the case of a TEMP location if it fails then the path on the "new" drive could be used for recovery before deletion and the subst deleted. Of course it cannot help someone who already has 23 drive letters mapped A: B: & C: are reserved! In the case of a virtual environment facing the issue I think that the best that can be done is to issue the user an error with a suggestion that they make the virtual environment with --relocatable and either move it higher up the directory tree or use subst to put it at the top of a "new" drive. I have just done a quick test on a Windows 10 (Anniversary Update) 64 bit machine and it would quite happily allow me to create a directory 217 characters long, (at which point the next mkdir failed and then mount it with 'subst' to a new drive letter and create a path 117 characters long on that drive - all that I had time for.
Contributor Author

### zooba commented Oct 27, 2016

 FWIW, Windows 10 allows the option to disable MAX_PATH limits for "self-certified" applications, which will include Python 3.6. Currently there's a machine policy that needs to be enabled, and the Python 3.6 installer will prompt users to do that (if they can - need to be admin). So the issue will go away in the future, and anything else we do now is mitigation for people on existing setups (which obviously has a lot of value, since it's going to be a long time before everyone is on Python 3.6 on Windows 10).
Contributor

### GadgetSteve commented Oct 29, 2016

 @zooba - My reading so far suggests that the disable MAX_PATH limits option is only planned to be available or possibly only accessible in Widows 10 Pro and Enterprise editions not the home edition.
Contributor Author

### zooba commented Oct 31, 2016 • edited

 @GadgetSteve Where are you reading that? It's highly unlikely that such core APIs would be different between Windows SKUs, though of course there's no support included in the home editions for managing group policy. Shouldn't prevent the registry edits from working though.
Contributor

### GadgetSteve commented Oct 31, 2016

 @zooba It looks like the only way to enable the change for the user is via the Group Policy Editor which is missing from the Home editions - I have had too many changes that should work via registry edits fail to work, revert on a MS Update or work erratically, (e.g. I cannot get the numb-lock to survive a reboot on Win 10 even after the registry edits), to be completely happy with the idea of relying on them.

Closed

Merged

Closed

Member

### pradyunsg commented Aug 21, 2017

 @pfmoore Any ideas on what could be a possible workaround for this?
Contributor

### GadgetSteve commented Aug 21, 2017

 On 21/08/2017 11:29, Pradyun Gedam wrote: @pfmoore Any ideas on what could be a possible workaround for this? If you are on Windows 10 applying the changes detailed at https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/ is a very good option. Otherwise options include: Installing python to a short path, e.g. C:\Py36 for Python 3.6 and setting temp to c:\tmp can sometimes help. If the path that is too long is the cashe path you can use pip --cache-dir=C:\PyCashe and the --root and --prefix options can also be helpful and sometimes you can make a long path into a short one with the command line subst and mklink commands. Of course there is always the option to switch to Linux... -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. … --- This email has been checked for viruses by AVG. http://www.avg.com
Contributor

### GadgetSteve commented Aug 21, 2017

 On 31/10/2016 16:52, Steve Dower wrote: @GadgetSteve Where are you reading that? It's highly unlikely that such core APIs would be different between Windows versions, though of course there's no support included in the home editions for managing group policy. Shouldn't prevent the registry edits from working though. The registry edits do work the early story that I was reading claimed that the fix would not be available other than via the group policy editor. … -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com

Closed

Member

### pradyunsg commented Aug 21, 2017 • edited

 Thanks @GadgetSteve! I'll try to summarise what I've understood. Here's how one can workaround this issue as of today: If you can use Python 3.6 and Windows 10, use them. Along with everything else, it has support for the long paths - since the Anniversary Update of Windows 10. You might have to enable long path support on Windows 10 though. [instructions] If you don't have the ability to use Windows 10 and Python 3.6, you'll have to workin the restrictions set by Windows: You can use the --cache-dir option to set a cache directory that's shorter. Set TEMP as a shorter path - like C:\Temp You can try making a long path into a short one with the command line tools subst and mklink commands. (but don't)
Member

### pradyunsg commented Aug 21, 2017

 Since this is basically fixed with the latest versions of the software involved (pip 9.0.1, Python 3.6.1, Windows 10 with Anniversary) - should there be changes in pip to workaround this issue?

### pfmoore added the R: deferred till PR label Aug 21, 2017

Contributor Author

### zooba commented Aug 21, 2017

 I wouldn't suggest going down the subst or mklink routes at all. pip should just name its temp directory something other than the full path to the eventual/original install location. The fix in Python 3.6 and Windows 10 is fine, but will never apply to all users.
Contributor

### GadgetSteve commented Aug 21, 2017

 Of course some deeply nested libraries may well end up hitting the problem with the actual files final location for user installs of python which is problematic, even if it never gets that far with the current process, maybe we also need a separate ticket for getting pip to auto-install to wheel with no override if platform is Windows and any of the resulting paths would exceed the limit.
Member

### pfmoore commented Aug 21, 2017

 I agree with @zooba - that's the form I'd expect such a patch to take. If pip can't install to the final location (as @GadgetSteve suggests) that's not an issue for pip - we'll get an OS error and roll back the install, and it's then the user's issue to solve.

Member

### pradyunsg commented Aug 21, 2017

 should there be changes in pip to workaround this issue? Yes - a patch is needed that shortens pip's temporary directory paths. FTR, "awaiting PR" is essentially for indicating that further discussion related to this issue should be deferred until someone comes around to make a PR.

Member

### pfmoore commented Aug 21, 2017

 Ah, OK. I've removed the label in that case, as there's no real need for any discussion. Do we not have a label for "Agreed to be a reasonable request, but won't go further until someone steps up with a PR"?
Member

### pradyunsg commented Aug 22, 2017

 @pfmoore Nope. Now that I think of it, the only reason I don't wanna use that label is because I have used it in a different context. I've mostly used a comment after labelling to describe the state of the issue; so, stating that you just need to make the PR would probably be fine here too. So, I dent mind if this gets labeled as awaiting PR. Maybe we should rename the current label to "deferred till PR" and add a new label with the current name for this situation? … On Tue, Aug 22, 2017, 03:01 Paul Moore ***@***.***> wrote: Ah, OK. I've removed the label in that case, as there's no real need for any discussion. Do we not have a label for "Agreed to be a reasonable request, but won't go further until someone steps up with a PR"? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3055 (comment)>, or mute the thread .
Member

### pradyunsg commented Aug 22, 2017

 Maybe we should rename the current label to "deferred till PR" and add a new label with the current name for this situation? I've gone ahead and done this.

Member

### pradyunsg commented Aug 22, 2017

 pip should just name its temp directory something other than the full path to the eventual/original install location. Noting for whomsoever comes around to making a PR for this, the behaviour change would be to shorten pip's temporary directories by not being the full path to the install location.

Open

Closed

Contributor

### GadgetSteve commented Jan 13, 2018

 On 27/10/2016 17:48, Steve Dower wrote: FWIW, Windows 10 allows the option to disable MAX_PATH limits for "self-certified" applications, which will include Python 3.6. Currently there's a machine policy that needs to be enabled, and the Python 3.6 installer will prompt users to do that (if they can - need to be admin). So the issue will go away in the future, and anything else we do now is mitigation for people on existing setups (which obviously has a lot of value, since it's going to be a long time before /everyone/ is on Python 3.6 on Windows 10). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3055 (comment)>, or mute the thread . Note that selecting this option enables long paths for everything, including python 2.7, and AFAIK it remains enabled even if the version of python that enabled it is subsequently removed. … -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.

### alexsilva commented Jan 30, 2018

 Path Length Limitation "\\?\D:\very long path" Note The maximum path of 32,767 characters is approximate, because the "\?" prefix may be expanded to a longer string by the system at run time, and this expansion applies to the total length.

Closed

Closed

### zooba referenced this issue Nov 21, 2018

Open

#### pip uninstall cannot create file in default TEMP path #6019

Contributor Author

### zooba commented Nov 21, 2018

 Question for @pradyunsg @pfmoore @dstufft: would you consider a change to try and use a random name in the target directory instead of TEMP? This has the nice advantage that we can simply rename a directory and then try to delete it, rather than doing multiple copies. When used on install, we can extract/install to the random name and then just rename the directory when it looks good, which drastically simplifies the process of copying permissions/etc., as they'll be inherited normally. And if the user doesn't have permissions to install, it'll fail much sooner. In both cases, the random name can be the same length as the target name, which will ensure that path length issues don't get any worse than when the package is installed. I guess the downside is potentially leftover files in the install directory when pip hard crashes, but the upside is less crashing and significantly faster installs (I've prototyped some parts of this on Windows and we're talking at least 2x faster for big packages like Django). Thoughts? Given one-off installs into virtual environments are becoming the norm, I think the risk of cruft being left behind in install directories is worth the other benefits.
Member

### dstufft commented Nov 21, 2018

 Seems reasonable to me, I'd just say we should use a name that isn't importable as well.
Contributor Author

### zooba commented Nov 21, 2018

 Substituting the first character with a digit is probably an easy algorithm, at least for the first 10 attempts :)
Member

### dstufft commented Nov 21, 2018

 Yea, or add a prefix like -pip-tmp or something.
Contributor Author

### zooba commented Nov 21, 2018

 I want to avoid generating a longer name, if possible. If we cross the 260 char barrier here, install will fail when the package would otherwise work (apart from caching pyc files... but oh well).
Member

### dstufft commented Nov 21, 2018

 Ah right, that makes sense. Okay.
Member

### dstufft commented Nov 21, 2018

 I think I'd use a leading - or something instead of a digit, just because it feel more purposeful to me? But that's kind of nitpicky, so just a suggestion.
Contributor Author

### zooba commented Nov 21, 2018

 I think it'll have to be an invalid character in a dist-info folder name as well, since those will need the same treatment. So probably I'm going to end up with a list of invalid package name characters that are valid directory name characters and swap them in until there's a free one. Trying - (or ~?) first is fine by me.
Member

### dstufft commented Nov 21, 2018

 Presumably another option here is to just not put the entire path inside of the temporary directory as well? Like if there's some common prefix here, we should be able to just exclude that from the temporary directory path that we generate so we're not nesting things nearly as far. That would turn something like: C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\users\\trevorsullivan\\source\\repos\\pythonapplication11\\pythonapplication11\\env\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc into C:\\Users\\TREVOR~1\\AppData\\Local\\Temp\\pip-h64zdfhc-uninstall\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\_implementation.cpython-34.pyc It doesn't have the other benefits though (simple renames, etc) and it means it could still fail when this strategy works in the case the TMPDIR path is longer than the common prefix we'd remove. On the flip side, it does mean that it's probably a smaller delta in our current behavior which may be a "safer" change to make. I don't feel strongly one way or another, but the idea popped into my head so figured I'd mention it.
Contributor Author

### zooba commented Nov 21, 2018

 Yeah, that is going to deal with the original problem here (most of the time). I think it was suggested a few times higher on the thread, and it's definitely a much simpler change. That said, I think the uninstall case is going to be just as simple to rename and then rmtree, so the delta doesn't get much bigger. And the perf benefit to wheel extraction is worth doing in-place as well. Maybe I'll start with uninstall though and send two separate PRs.
Member

### pfmoore commented Nov 21, 2018

 I'm OK with the idea of a random name in the target. Just to be clear, we're talking about the temporary install directory that gets moved into place for the final install? Not the build directory?
Member

### pradyunsg commented Nov 21, 2018

 Sounds OK to me as well.
Contributor Author

### zooba commented Nov 21, 2018

 Just to be clear, we're talking about the temporary install directory that gets moved into place for the final install? Not the build directory? Correct. IIUC, eventually all installs will come from/via wheels, right? So the install side will only affect the wheel extraction directory, and shouldn't touch build at all.
Member

### pfmoore commented Nov 21, 2018

 Yep, under PEP 517 (now in master, but not yet used for all builds) we set up a build directory (which will still be in \$TEMP) and call the backend to build a wheel. We then extract the wheel and move the extracted files. If we extract to a random-named directory in the target location and move, that will skip a whole copy step, which is a definite performance improvement[1] Legacy non-PEP517 installs go direct via setup.py install, so there's no wheel or wheel extraction step, so they won't change (which is fine by me, as you say that code path is destined for removal anyway) [1] We might get a double improvement - I don't know much about how anti-virus software works, but if it can recognise that a move doesn't require a new scan, that could mean that the proposed approach will remove an extra unneeded virus scan as well.
Contributor Author

### zooba commented Nov 21, 2018

 Pretty sure we'll get that double improvement :) I've been profiling with AV scanning enabled, and it seems to be triggered totally differently between copyfile (open/read/close/open/write/close) and rename.

Merged

Open

### pfmoore added a commit that referenced this issue Dec 6, 2018

Merge pull request #6029 from zooba/issue-3055
Fixes #3055 Uninstall causes paths to exceed MAX_PATH limit

### borsbot added a commit to mozilla/normandy that referenced this issue Feb 7, 2019

Merge #1723
1723: Scheduled weekly dependency update for week 05 r=mythmon a=pyup-bot

### Update [atomicwrites](https://pypi.org/project/atomicwrites) from **1.2.1** to **1.3.0**.

*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>

- PyPI: https://pypi.org/project/atomicwrites
- Repo: https://github.com/untitaker/python-atomicwrites
</details>

### Update [botocore](https://pypi.org/project/botocore) from **1.12.82** to **1.12.86**.

<details>
<summary>Changelog</summary>

### 1.12.86

=======

* api-change:devicefarm: Update devicefarm client to latest version
* api-change:codecommit: Update codecommit client to latest version
* api-change:medialive: Update medialive client to latest version
* api-change:mediaconnect: Update mediaconnect client to latest version


### 1.12.85

=======

* api-change:logs: Update logs client to latest version
* api-change:elbv2: Update elbv2 client to latest version
* api-change:rds: Update rds client to latest version
* api-change:codebuild: Update codebuild client to latest version
* api-change:sms-voice: Update sms-voice client to latest version
* api-change:ecr: Update ecr client to latest version


### 1.12.84

=======

* api-change:worklink: Update worklink client to latest version
* api-change:apigatewaymanagementapi: Update apigatewaymanagementapi client to latest version
* api-change:acm-pca: Update acm-pca client to latest version


### 1.12.83

=======

* api-change:appstream: Update appstream client to latest version
* api-change:discovery: Update discovery client to latest version
* api-change:dms: Update dms client to latest version
* api-change:fms: Update fms client to latest version
* api-change:ssm: Update ssm client to latest version


</details>

<details>

- PyPI: https://pypi.org/project/botocore
- Changelog: https://pyup.io/changelogs/botocore/
- Repo: https://github.com/boto/botocore
</details>

### Update [Faker](https://pypi.org/project/Faker) from **1.0.1** to **1.0.2**.

<details>
<summary>Changelog</summary>

### 1.0.2

--------------------------------------------------------------------------------------

* Fix state abbreviations for id_ID to be 2-letters. Thanks dt-ap.
* Fix format for city_with_postcode on de_DE locale. Thanks TZanke.
* Update person providers for zh_CN. Thanks TimeFinger.
* Implement zipcode_in_state and aliases in en_US locale for generating
a zipcode for a specified state. Thanks mattyg.
* Group first names by gender on zh_CN provider. Thanks TimeFinger.


</details>

<details>

- PyPI: https://pypi.org/project/faker
- Changelog: https://pyup.io/changelogs/faker/
- Repo: https://github.com/joke2k/faker
</details>

### Update [pycodestyle](https://pypi.org/project/pycodestyle) from **2.4.0** to **2.5.0**.

<details>
<summary>Changelog</summary>

### 2.5.0

------------------

New checks:

* E117: Over-indented code blocks
* W505: Maximum doc-string length only when configured with --max-doc-length

Changes:

* Remove support for EOL Python 2.6 and 3.3. PR 720.
* Add E117 error for over-indented code blocks.
* Allow W605 to be silenced by  noqa and fix the position reported by W605
* Allow users to omit blank lines around one-liner definitions of classes and
functions
* Include the function return annotation (-&gt;) as requiring surrounding
whitespace only on Python 3
* Verify that only names can follow await. Previously we allowed numbers
and strings.
* Add support for Python 3.7
* Fix detection of annotated argument defaults for E252
* Cprrect the position reported by W504


</details>

<details>

- PyPI: https://pypi.org/project/pycodestyle
- Changelog: https://pyup.io/changelogs/pycodestyle/
</details>

### Update [pyflakes](https://pypi.org/project/pyflakes) from **2.0.0** to **2.1.0**.

<details>
<summary>Changelog</summary>

### 2.1.0

- Allow intentional assignment to variables named _
- Recognize __module__ as a valid name in class scope
- pyflakes.checker.Checker supports checking of partial ast trees
- Detect assign-before-use for local variables which shadow builtin names
- Detect invalid print syntax using &gt;&gt; operator
- Treat async for the same as a for loop for introducing variables
- Add detection for list concatenation in __all__
- Exempt typing.overload from duplicate function declaration
- Importing a submodule of an as-aliased import-import is marked as
used
- Report undefined names from __all__ as possibly coming from a *
import
- Add support for changes in Python 3.8-dev
- Add support for PEP 563 (from __future__ import annotations)
- Include Python version and platform information in pyflakes --version
- Recognize __annotations__ as a valid magic global in Python 3.6+
- Mark names used in PEP 484  type: ... comments as used
- Add check for use of is operator with str, bytes, and int
literals


</details>

<details>

- PyPI: https://pypi.org/project/pyflakes
- Changelog: https://pyup.io/changelogs/pyflakes/
- Repo: https://github.com/PyCQA/pyflakes
</details>

### Update [setuptools](https://pypi.org/project/setuptools) from **40.6.3** to **40.7.3**.

<details>
<summary>Changelog</summary>

### 40.7.3

-------

* 1670: In package_index, revert to using a copy of splituser from Python 3.8. Attempts to use urllib.parse.urlparse led to problems as reported in 1663 and 1668. This change serves as an alternative to 1499 and fixes 1668.


### 40.7.2

-------

* 1666: Restore port in URL handling in package_index.


### 40.7.1

-------

* 1660: On Python 2, when reading config files, downcast options from text to bytes to satisfy distutils expectations.


### 40.7.0

-------

* 1551: File inputs for the license field in setup.cfg files now explicitly raise an error.
* 1180: Add support for non-ASCII in setup.cfg (1062). Add support for native strings on some parameters (1136).
* 1499: setuptools.package_index no longer relies on the deprecated urllib.parse.splituser per Python 27485.
* 1625: In PEP 517 build_meta builder, ensure that sdists are built as gztar per the spec.


</details>

<details>

- PyPI: https://pypi.org/project/setuptools
- Changelog: https://pyup.io/changelogs/setuptools/
- Repo: https://github.com/pypa/setuptools
</details>

### Update [cachetools](https://pypi.org/project/cachetools) from **3.0.0** to **3.1.0**.

<details>
<summary>Changelog</summary>

### 3.1.0

-------------------

- Fix Python 3.8 compatibility issue.

- Use time.monotonic as default timer if available.

- Improve documentation regarding thread safety.


</details>

<details>

- PyPI: https://pypi.org/project/cachetools
- Changelog: https://pyup.io/changelogs/cachetools/
- Repo: https://github.com/tkem/cachetools
</details>

### Update [boto3](https://pypi.org/project/boto3) from **1.9.82** to **1.9.86**.

<details>
<summary>Changelog</summary>

### 1.9.86

======

* api-change:devicefarm: [botocore] Update devicefarm client to latest version
* api-change:codecommit: [botocore] Update codecommit client to latest version
* api-change:medialive: [botocore] Update medialive client to latest version
* api-change:mediaconnect: [botocore] Update mediaconnect client to latest version


### 1.9.85

======

* api-change:logs: [botocore] Update logs client to latest version
* api-change:elbv2: [botocore] Update elbv2 client to latest version
* api-change:rds: [botocore] Update rds client to latest version
* api-change:codebuild: [botocore] Update codebuild client to latest version
* api-change:sms-voice: [botocore] Update sms-voice client to latest version
* api-change:ecr: [botocore] Update ecr client to latest version


### 1.9.84

======

* api-change:worklink: [botocore] Update worklink client to latest version
* api-change:apigatewaymanagementapi: [botocore] Update apigatewaymanagementapi client to latest version
* api-change:acm-pca: [botocore] Update acm-pca client to latest version


### 1.9.83

======

* api-change:appstream: [botocore] Update appstream client to latest version
* api-change:discovery: [botocore] Update discovery client to latest version
* api-change:dms: [botocore] Update dms client to latest version
* api-change:fms: [botocore] Update fms client to latest version
* api-change:ssm: [botocore] Update ssm client to latest version


</details>

<details>

- PyPI: https://pypi.org/project/boto3
- Changelog: https://pyup.io/changelogs/boto3/
- Repo: https://github.com/boto/boto3
</details>

### Update [flake8](https://pypi.org/project/flake8) from **3.6.0** to **3.7.5**.

<details>
<summary>Changelog</summary>

### 3.7.5

-------------------

You can view the 3.7.5 milestone_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

GitLab!301_, GitLab503_)

.. _3.7.5 milestone:
https://gitlab.com/pycqa/flake8/milestones/28

.. _GitLab503:
https://gitlab.com/pycqa/flake8/issues/503

.. _GitLab!301:
https://gitlab.com/pycqa/flake8/merge_requests/301


### 3.7.4

-------------------

You can view the 3.7.4 milestone_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix performance regression with lots of per-file-ignores and errors
(See also GitLab!299_, GitLab501_)

.. _3.7.4 milestone:
https://gitlab.com/pycqa/flake8/milestones/27

.. _GitLab501:
https://gitlab.com/pycqa/flake8/issues/501

.. _GitLab!299:
https://gitlab.com/pycqa/flake8/merge_requests/299


### 3.7.3

-------------------

You can view the 3.7.3 milestone_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix imports of typing in python 3.5.0 / 3.5.1 (See also GitLab!294_,
GitLab498_)

- Fix flake8 --statistics (See also GitLab!295_, GitLab499_)

- Gracefully ignore flake8-per-file-ignores plugin if installed (See also
GitLab!297_, GitLab495_)

- Improve error message for malformed per-file-ignores (See also
GitLab!298_, GitLab489_)

.. _3.7.3 milestone:
https://gitlab.com/pycqa/flake8/milestones/26

.. _GitLab489:
https://gitlab.com/pycqa/flake8/issues/489
.. _GitLab495:
https://gitlab.com/pycqa/flake8/issues/495
.. _GitLab498:
https://gitlab.com/pycqa/flake8/issues/498
.. _GitLab499:
https://gitlab.com/pycqa/flake8/issues/499

.. _GitLab!294:
https://gitlab.com/pycqa/flake8/merge_requests/294
.. _GitLab!295:
https://gitlab.com/pycqa/flake8/merge_requests/295
.. _GitLab!297:
https://gitlab.com/pycqa/flake8/merge_requests/297
.. _GitLab!298:
https://gitlab.com/pycqa/flake8/merge_requests/298


### 3.7.2

-------------------

You can view the 3.7.2 milestone_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix broken flake8 --diff (regressed in 3.7.0) (See also GitLab!292_,
GitLab490_)

- Fix typo in plugin exception reporting (See also GitLab!275_,
GitLab491_)

- Fix AttributeError while attempting to use the legacy api (regressed in
3.7.0) (See also GitLab!293_, GitLab497_)

.. _3.7.2 milestone:
https://gitlab.com/pycqa/flake8/milestones/25

.. _GitLab490:
https://gitlab.com/pycqa/flake8/issues/490
.. _GitLab491:
https://gitlab.com/pycqa/flake8/issues/491
.. _GitLab497:
https://gitlab.com/pycqa/flake8/issues/497

.. _GitLab!292:
https://gitlab.com/pycqa/flake8/merge_requests/292
.. _GitLab!275:
https://gitlab.com/pycqa/flake8/merge_requests/275
.. _GitLab!293:
https://gitlab.com/pycqa/flake8/merge_requests/293


### 3.7.1

-------------------

You can view the 3.7.1 milestone_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix capitalized filenames in per-file-ignores setting (See also
GitLab!290_, GitLab488_)

.. _3.7.1 milestone:
https://gitlab.com/pycqa/flake8/milestones/24

.. _GitLab488:
https://gitlab.com/pycqa/flake8/issues/488

.. _GitLab!290:
https://gitlab.com/pycqa/flake8/merge_requests/290


### 3.7.0

-------------------

You can view the 3.7.0 milestone_ on GitLab for more details.

New Dependency Information
~~~~~~~~~~~~~~~~~~~~~~~~~~

- Add dependency on entrypoints &gt;= 0.3, &lt; 0.4 (See also GitLab!264_,
GitLab!288_)

- Pyflakes has been updated to &gt;= 2.1.0, &lt; 2.2.0 (See also GitLab!283_,
GitLab!285_)

- pycodestyle has been updated to &gt;= 2.5.0, &lt; 2.6.0 (See also GitLab!287_)

Features
~~~~~~~~

- Add support for per-file-ignores (See also GitLab!259_, GitLab156_,
GitLab!281_, GitLab471_)

- Enable use of float and complex option types (See also GitLab!261_,
GitLab452_)

- Improve startup performance by switching from pkg_resources to
entrypoints (See also GitLab!264_)

- Add metadata for use through the pre-commit_ git hooks framework (See also
GitLab!268_, GitLab!284_)

- Allow physical line checks to return more than one result (See also
GitLab!269_)

- Allow  noqa:X123 comments without space between the colon and codes
list (See also GitLab!273_, GitLab470_)

- Remove broken and unused flake8.listen plugin type (See also
GitLab!274_, GitLab480_)

.. _3.7.0 milestone:
https://gitlab.com/pycqa/flake8/milestones/23
.. _pre-commit:
https://pre-commit.com/

.. _GitLab156:
https://gitlab.com/pycqa/flake8/issues/156
.. _GitLab452:
https://gitlab.com/pycqa/flake8/issues/452
.. _GitLab470:
https://gitlab.com/pycqa/flake8/issues/470
.. _GitLab471:
https://gitlab.com/pycqa/flake8/issues/471
.. _GitLab480:
https://gitlab.com/pycqa/flake8/issues/480

.. _GitLab!259:
https://gitlab.com/pycqa/flake8/merge_requests/259
.. _GitLab!261:
https://gitlab.com/pycqa/flake8/merge_requests/261
.. _GitLab!264:
https://gitlab.com/pycqa/flake8/merge_requests/264
.. _GitLab!268:
https://gitlab.com/pycqa/flake8/merge_requests/268
.. _GitLab!269:
https://gitlab.com/pycqa/flake8/merge_requests/269
.. _GitLab!273:
https://gitlab.com/pycqa/flake8/merge_requests/273
.. _GitLab!274:
https://gitlab.com/pycqa/flake8/merge_requests/274
.. _GitLab!281:
https://gitlab.com/pycqa/flake8/merge_requests/281
.. _GitLab!283:
https://gitlab.com/pycqa/flake8/merge_requests/283
.. _GitLab!284:
https://gitlab.com/pycqa/flake8/merge_requests/284
.. _GitLab!285:
https://gitlab.com/pycqa/flake8/merge_requests/285
.. _GitLab!287:
https://gitlab.com/pycqa/flake8/merge_requests/287
.. _GitLab!288:
https://gitlab.com/pycqa/flake8/merge_requests/288


</details>

<details>

- PyPI: https://pypi.org/project/flake8
- Changelog: https://pyup.io/changelogs/flake8/
- Repo: https://gitlab.com/pycqa/flake8
</details>

### Update [newrelic](https://pypi.org/project/newrelic) from **4.10.0.112** to **4.12.0.113**.

<details>
<summary>Changelog</summary>

### 4.12.0.113

This release of the Python agent extends support of Amazon&#39;s boto3 library and includes bug fixes.

The agent can be installed using easy_install/pip/distribute via the Python Package Index or can be downloaded directly from the New Relic download site.

Features

AWS operation and request ID will now be reported in transaction traces and
spans when using boto3 and botocore.

The agent will now report aws.requestId and aws.operation for all calls
to AWS made using botocore and boto3.
DynamoDB calls are now reported under the Databases tab.

The agent will now record DynamoDB query performance in the Databases tab in
APM in addition to table name for the following calls:

put_item
get_item
update_item
delete_item
create_table
delete_table
query
scan

traces.

The agent will now record the queue name in spans and transaction traces for
the following SQS calls:

send_message
send_message_batch

The SNS topic, target, or the string literal PhoneNumber will be reported to
New Relic inside of spans and transaction traces.
The full URL path will now be recorded on span events and transaction traces
when using boto3 or botocore.

The agent will now record the full URL path for API calls made to AWS through
the boto3 / botocore libraries. The path will be available through span
events and transaction traces.

Bug Fixes

Using newrelic-admin to start a GunicornWebWorker with an application factory
resulted in an application crash.

The agent would fail to start if using the newrelic-admin command to start an
aiohttp application factory with GunicornWebWorker. This issue has now been
fixed.


</details>

<details>

- PyPI: https://pypi.org/project/newrelic
- Changelog: https://pyup.io/changelogs/newrelic/
- Homepage: http://newrelic.com/docs/python/new-relic-for-python
</details>

### Update [psycopg2](https://pypi.org/project/psycopg2) from **2.7.6.1** to **2.7.7**.

<details>
<summary>Changelog</summary>

### 2.7.7

^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Cleanup of the cursor results assignment code, which might have solved
double free and inconsistencies in concurrent usage (🎟346, 384).
- Wheel package compiled against OpenSSL 1.0.2q.


</details>

<details>

- PyPI: https://pypi.org/project/psycopg2
- Changelog: https://pyup.io/changelogs/psycopg2/
- Homepage: http://initd.org/psycopg/
</details>

### Update [pyasn1-modules](https://pypi.org/project/pyasn1-modules) from **0.2.3** to **0.2.4**.

*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>

- PyPI: https://pypi.org/project/pyasn1-modules
- Changelog: https://pyup.io/changelogs/pyasn1-modules/
- Repo: https://github.com/etingof/pyasn1-modules
</details>

### Update [pytest-django](https://pypi.org/project/pytest-django) from **3.4.5** to **3.4.7**.

<details>
<summary>Changelog</summary>

### 3.4.7

------------------

Bugfixes
^^^^^^^^

* Fix disabling/handling of unittest methods with pytest 4.2+ (700)


### 3.4.6

------------------

Bugfixes
^^^^^^^^

* django_find_project: add cwd as fallback always (690)

Misc
^^^^

* Enable tests for Django 2.2 and add classifier (693)
* Disallow pytest 4.2.0 in install_requires (697)


</details>

<details>

- PyPI: https://pypi.org/project/pytest-django
- Changelog: https://pyup.io/changelogs/pytest-django/
</details>

### Update [pytest](https://pypi.org/project/pytest) from **4.1.1** to **4.2.0**.

<details>
<summary>Changelog</summary>

### 4.2.0

=========================

Features
--------

- 3094 &lt;https://github.com/pytest-dev/pytest/issues/3094&gt;_: Class xunit-style &lt;https://docs.pytest.org/en/latest/xunit_setup.html&gt;__ functions and methods
now obey the scope of *autouse* fixtures.

This fixes a number of surprising issues like setup_method being called before session-scoped
autouse fixtures (see 517 &lt;https://github.com/pytest-dev/pytest/issues/517&gt;__ for an example).

- 4627 &lt;https://github.com/pytest-dev/pytest/issues/4627&gt;_: Display a message at the end of the test session when running under Python 2.7 and 3.4 that pytest 5.0 will no longer
support those Python versions.

- 4660 &lt;https://github.com/pytest-dev/pytest/issues/4660&gt;_: The number of *selected* tests now are also displayed when the -k or -m flags are used.

- 4688 &lt;https://github.com/pytest-dev/pytest/issues/4688&gt;_: pytest_report_teststatus hook now can also receive a config parameter.

- 4691 &lt;https://github.com/pytest-dev/pytest/issues/4691&gt;_: pytest_terminal_summary hook now can also receive a config parameter.

Bug Fixes
---------

- 3547 &lt;https://github.com/pytest-dev/pytest/issues/3547&gt;_: --junitxml can emit XML compatible with Jenkins xUnit.
junit_family INI option accepts legacy|xunit1, which produces old style output, and xunit2 that conforms more strictly to https://github.com/jenkinsci/xunit-plugin/blob/xunit-2.3.2/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd

- 4280 &lt;https://github.com/pytest-dev/pytest/issues/4280&gt;_: Improve quitting from pdb, especially with --trace.

Using q[quit] after pdb.set_trace() will quit pytest also.

- 4402 &lt;https://github.com/pytest-dev/pytest/issues/4402&gt;_: Warning summary now groups warnings by message instead of by test id.

This makes the output more compact and better conveys the general idea of how much code is
actually generating warnings, instead of how many tests call that code.

- 4536 &lt;https://github.com/pytest-dev/pytest/issues/4536&gt;_: monkeypatch.delattr handles class descriptors like staticmethod/classmethod.

- 4649 &lt;https://github.com/pytest-dev/pytest/issues/4649&gt;_: Restore marks being considered keywords for keyword expressions.

- 4653 &lt;https://github.com/pytest-dev/pytest/issues/4653&gt;_: tmp_path fixture and other related ones provides resolved path (a.k.a real path)

- 4667 &lt;https://github.com/pytest-dev/pytest/issues/4667&gt;_: pytest_terminal_summary uses result from pytest_report_teststatus hook, rather than hardcoded strings.

- 4669 &lt;https://github.com/pytest-dev/pytest/issues/4669&gt;_: Correctly handle unittest.SkipTest exception containing non-ascii characters on Python 2.

- 4680 &lt;https://github.com/pytest-dev/pytest/issues/4680&gt;_: Ensure the tmpdir and the tmp_path fixtures are the same folder.

- 4681 &lt;https://github.com/pytest-dev/pytest/issues/4681&gt;_: Ensure tmp_path is always a real path.

Trivial/Internal Changes
------------------------

- 4643 &lt;https://github.com/pytest-dev/pytest/issues/4643&gt;_: Use a.item() instead of the deprecated np.asscalar(a) in pytest.approx.

np.asscalar has been deprecated &lt;https://github.com/numpy/numpy/blob/master/doc/release/1.16.0-notes.rstnew-deprecations&gt;__ in numpy 1.16..

- 4657 &lt;https://github.com/pytest-dev/pytest/issues/4657&gt;_: Copy saferepr from pylib


</details>

<details>

- PyPI: https://pypi.org/project/pytest
- Changelog: https://pyup.io/changelogs/pytest/
- Homepage: https://docs.pytest.org/en/latest/
</details>

### Update [pytest-mock](https://pypi.org/project/pytest-mock) from **1.10.0** to **1.10.1**.

*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>

- PyPI: https://pypi.org/project/pytest-mock
- Changelog: https://pyup.io/changelogs/pytest-mock/
- Repo: https://github.com/pytest-dev/pytest-mock/
</details>

### Update [Sphinx](https://pypi.org/project/Sphinx) from **1.8.3** to **1.8.4**.

<details>
<summary>Changelog</summary>

### 1.8.4

=====================================

Bugs fixed
----------

* 3707: latex: no bold checkmark (✔) available.
* 5605: with the documentation language set to Chinese, English words could not
be searched.
* 5889: LaTeX: user numfig_format is stripped of spaces and may cause
build failure
* C++, fix hyperlinks for declarations involving east cv-qualifiers.
* 5755: C++, fix duplicate declaration error on function templates with constraints
in the return type.
* C++, parse unary right fold expressions and binary fold expressions.
* pycode could not handle egg files on windows
* 5928: KeyError: &#39;DOCUTILSCONFIG&#39; when running build
* 5936: LaTeX: PDF build broken by inclusion of image taller than page height
* 5231: &quot;make html&quot; does not read and build &quot;po&quot; files in &quot;locale&quot; dir
* 5954: :scale: image option may break PDF build if image in an admonition
* 5966: mathjax has not been loaded on incremental build
* 5960: LaTeX: modified PDF layout since September 2018 TeXLive update of
:file:parskip.sty
* 5948: LaTeX: duplicated labels are generated for sections
* 5958: versionadded directive causes crash with Python 3.5.0
* 5995: autodoc: autodoc_mock_imports conflict with metaclass on Python 3.7
* 5871: texinfo: a section title . is not allowed


</details>

<details>

- PyPI: https://pypi.org/project/sphinx
- Changelog: https://pyup.io/changelogs/sphinx/
- Homepage: http://sphinx-doc.org/
</details>

### Update [pip](https://pypi.org/project/pip) from **18.1** to **19.0.1**.

<details>
<summary>Changelog</summary>

### 19.0

=================

Deprecations and Removals
-------------------------

- Deprecate support for Python 3.4 (6106 &lt;https://github.com/pypa/pip/issues/6106&gt;_)
- Start printing a warning for Python 2.7 to warn of impending Python 2.7 End-of-life and
prompt users to start migrating to Python 3. (6148 &lt;https://github.com/pypa/pip/issues/6148&gt;_)
- Remove the deprecated --process-dependency-links option. (6060 &lt;https://github.com/pypa/pip/issues/6060&gt;_)
- Remove the deprecated SVN editable detection based on dependency links
during freeze. (5866 &lt;https://github.com/pypa/pip/issues/5866&gt;_)

Features
--------

- Implement PEP 517 (allow projects to specify a build backend via pyproject.toml). (5743 &lt;https://github.com/pypa/pip/issues/5743&gt;_)
- Implement manylinux2010 platform tag support.  manylinux2010 is the successor
to manylinux1.  It allows carefully compiled binary wheels to be installed
on compatible Linux platforms. (5008 &lt;https://github.com/pypa/pip/issues/5008&gt;_)
- Improve build isolation: handle .pth files, so namespace packages are correctly supported under Python 3.2 and earlier. (5656 &lt;https://github.com/pypa/pip/issues/5656&gt;_)
- Include the package name in a freeze warning if the package is not installed. (5943 &lt;https://github.com/pypa/pip/issues/5943&gt;_)
- Warn when dropping an --[extra-]index-url value that points to an existing local directory. (5827 &lt;https://github.com/pypa/pip/issues/5827&gt;_)
- Prefix pip&#39;s --log file lines with their timestamp. (6141 &lt;https://github.com/pypa/pip/issues/6141&gt;_)

Bug Fixes
---------

- Avoid creating excessively long temporary paths when uninstalling packages. (3055 &lt;https://github.com/pypa/pip/issues/3055&gt;_)
- Redact the password from the URL in various log messages. (4746 &lt;https://github.com/pypa/pip/issues/4746&gt;_, 6124 &lt;https://github.com/pypa/pip/issues/6124&gt;_)
- Avoid creating excessively long temporary paths when uninstalling packages. (3055 &lt;https://github.com/pypa/pip/issues/3055&gt;_)
- Avoid printing a stack trace when given an invalid requirement. (5147 &lt;https://github.com/pypa/pip/issues/5147&gt;_)
- Present 401 warning if username/password do not work for URL (4833 &lt;https://github.com/pypa/pip/issues/4833&gt;_)
- Handle requests.exceptions.RetryError raised in PackageFinder that was causing pip to fail silently when some indexes were unreachable. (5270 &lt;https://github.com/pypa/pip/issues/5270&gt;_, 5483 &lt;https://github.com/pypa/pip/issues/5483&gt;_)
- Handle a broken stdout pipe more gracefully (e.g. when running pip list | head). (4170 &lt;https://github.com/pypa/pip/issues/4170&gt;_)
- Fix crash from setting PIP_NO_CACHE_DIR=yes. (5385 &lt;https://github.com/pypa/pip/issues/5385&gt;_)
- Fix crash from unparseable requirements when checking installed packages. (5839 &lt;https://github.com/pypa/pip/issues/5839&gt;_)
- Fix content type detection if a directory named like an archive is used as a package source. (5838 &lt;https://github.com/pypa/pip/issues/5838&gt;_)
- Fix listing of outdated packages that are not dependencies of installed packages in pip list --outdated --not-required (5737 &lt;https://github.com/pypa/pip/issues/5737&gt;_)
- Fix sorting TypeError in move_wheel_files() when installing some packages. (5868 &lt;https://github.com/pypa/pip/issues/5868&gt;_)
- Fix support for invoking pip using python src/pip .... (5841 &lt;https://github.com/pypa/pip/issues/5841&gt;_)
- Greatly reduce memory usage when installing wheels containing large files. (5848 &lt;https://github.com/pypa/pip/issues/5848&gt;_)
- Editable non-VCS installs now freeze as editable. (5031 &lt;https://github.com/pypa/pip/issues/5031&gt;_)
- Editable Git installs without a remote now freeze as editable. (4759 &lt;https://github.com/pypa/pip/issues/4759&gt;_)
- Canonicalize sdist file names so they can be matched to a canonicalized package name passed to pip install. (5870 &lt;https://github.com/pypa/pip/issues/5870&gt;_)
- Properly decode special characters in SVN URL credentials. (5968 &lt;https://github.com/pypa/pip/issues/5968&gt;_)
- Make PIP_NO_CACHE_DIR disable the cache also for truthy values like &quot;true&quot;, &quot;yes&quot;, &quot;1&quot;, etc. (5735 &lt;https://github.com/pypa/pip/issues/5735&gt;_)

Vendored Libraries
------------------

- Include license text of vendored 3rd party libraries. (5213 &lt;https://github.com/pypa/pip/issues/5213&gt;_)
- Update certifi to 2018.11.29
- Update colorama to 0.4.1
- Update distlib to 0.2.8
- Update idna to 2.8
- Update packaging to 19.0
- Update pep517 to 0.5.0
- Update pkg_resources to 40.6.3 (via setuptools)
- Update pyparsing to 2.3.1
- Update pytoml to 0.1.20
- Update requests to 2.21.0
- Update six to 1.12.0
- Update urllib3 to 1.24.1

Improved Documentation
----------------------

- Include the Vendoring Policy in the documentation. (5958 &lt;https://github.com/pypa/pip/issues/5958&gt;_)
- Add instructions for running pip from source to Development documentation. (5949 &lt;https://github.com/pypa/pip/issues/5949&gt;_)
- Remove references to removed egg=&lt;name&gt;-&lt;version&gt; functionality (5888 &lt;https://github.com/pypa/pip/issues/5888&gt;_)
- Fix omission of command name in HTML usage documentation (5984 &lt;https://github.com/pypa/pip/issues/5984&gt;_)


</details>

<details>

- PyPI: https://pypi.org/project/pip
- Changelog: https://pyup.io/changelogs/pip/
- Homepage: https://pip.pypa.io/
</details>

Co-authored-by: pyup-bot <github-bot@pyup.io>
Co-authored-by: Peter Bengtsson <mail@peterbe.com>