pip upgrade fails due to permissions error, then claims it didn't fail

[AMDphreak]

Environment

• pip version: 19.0.3 (or is it?...read below)
• Python version: 3.7.4 x86 or x64
• OS: Win 10 Pro x64 Version 10.0.18362 Build 18362

Description
Upgrading pip reports that it fails due to a permissions error, then re-running the upgrade reports that it was already installed.

Expected behavior
Either fail and complain or install and don't complain.

How to Reproduce

1. Run pip install --upgrade pip. See error.
2. Run pip install --upgrade pip a second time. Claims it was already installed.

Output

First upgrade fails:

C:\Users\rjamd>pip install --upgrade pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/8d/07/f7d7ced2f97ca3098c16565efbe6b15fafcba53e8d9bdb431e09140514b0/pip-19.2.2-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.0.3
    Uninstalling pip-19.0.3:
      Successfully uninstalled pip-19.0.3
Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'C:\\Users\\rjamd\\AppData\\Local\\Temp\\pip-uninstall-3x5sa0o8\\pip.exe'
Consider using the `--user` option or check the permissions.

Second upgrade claims it didn't fail the first time:

C:\Users\rjamd>pip install --upgrade pip
Requirement already up-to-date: pip in c:\users\rjamd\appdata\local\programs\python\python37\lib\site-packages (19.2.2)

C:\Users\rjamd>pip -V
pip 19.2.2 from c:\users\rjamd\appdata\local\programs\python\python37\lib\site-packages\pip (python 3.7)

The site-packages folder does not contain an exe called pip.exe but does contain a folder called pip, and another folder called pip-19.2.2.dist-info

This is where all the python's exe's are:

C:\Users\rjamd>where python
C:\Users\rjamd\AppData\Local\Programs\Python\Python37\python.exe
C:\Users\rjamd\AppData\Local\Programs\Python\Python37-32\python.exe
C:\Users\rjamd\AppData\Local\Microsoft\WindowsApps\python.exe

I checked the Temp folder and pip exists in it, but why would it have trouble deleting its own temp folder? I couldn't get any version info from the exe metadata, so I directly ran it in CMD and found out literally nothing, because it reports the version from a different directory, weirdly enough:

C:\Users\rjamd>C:\Users\rjamd\AppData\Local\Temp\pip-uninstall-3x5sa0o8\pip -V
pip 19.2.2 from c:\users\rjamd\appdata\local\programs\python\python37\lib\site-packages\pip (python 3.7)

[atugushev]

I believe this must be fixed after #6864 merge.

[JordonMMG]

Issue still exists:

[chrahunt]

So there are two issues here:

1. pip should be protecting itself from upgrade on Windows when invoked like pip (as opposed to python -m pip - this was fixed in Fix bypassed pip upgrade warning on Windows #6864 which is available in pip 19.3

2. when pip installs an existing package it does the following:

   1. uninstalls the existing package by moving the files off to the side (like the temp directory you mentioned)
   2. installs the new package
   3. if that succeeded, remove the old files
   4. print "Successfully installed" message

   Based on the error message, we failed at step 3. So technically the install did succeed, but our cleanup failed.

Given that pip itself created the temp directory, there's two situations that can cause this:

1. Pip uninstalling itself - as mentioned above by @atugushev this should be fixed
2. very long running external process keeping the file open should be - 7280 alternative fix: wrap existing rmtree to retry longer only for windows access errors. #7363 should help here

so we should be covered here.

I checked the Temp folder and pip exists in it, but why would it have trouble deleting its own temp folder?

On Windows, in general, you cannot delete files that are in use by other applications (even the same application!). pip.exe was being used by the pip process itself so it could not be deleted.

pip version: 19.0.3 (or is it?...read below)

The pip.exe doesn't directly contain the version information, but looks at the installed files. Since at that point the new pip was installed it outputs the version info for that one.

I hope that helps, please let me know if anything is unclear.

[chrahunt]

Hi @JordonMMG. That looks like a different issue. You're correctly using python -m pip to upgrade pip, but it looks like Python was installed into C:\Program Files (x86)\..., which you would not normally have write permission to unless you started an admin command prompt. Installing with --user is the right thing to do in that case.

Also I would note that it is kind of strange that pip prints a warning after you have just upgraded. If this was the first package that you've installed with --user, then the folder that pip gets installed into may not have existed so we wouldn't've been able to see the newly installed version to avoid printing the warning. If you had already installed packages with --user in that Python environment before then let me know and we can investigate further. Otherwise, I think we're doing the best we can in this particular case.

If you have any other problems, please feel free to file another issue and we can help!

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.