# windows in-place pip upgrades #1299

Closed
opened this Issue Nov 5, 2013 · 85 comments

Projects
None yet
Contributor

### qwcode commented Nov 5, 2013

 not sure of the status of pip install --ugrade pip on windows. need this for PEP453 see #188 for an old discussion on this.
Member

### pfmoore commented Nov 5, 2013

 Use python -m pip install --upgrade pip. I think @vsajip was considering some form of magic in the exe wrappers to mean that self-replacement of the exe could be made to work. But I don't know if he did anything in the version of distlib we're using. I'm not actually sure it's something that could be done without help from the application (pip). And I'm pretty sure no code has been added to pip for this. Personally, I consider this to be a minor issue - telling people to use python -m pip seems to me to be a perfectly acceptable answer, given that the issue is fundamental to how Windows handles deletion of open files. Ultimately it's going to be a question of how much complexity (with the consequent maintenance cost) and risk we want to take on in order to solve this.
Member

### dstufft commented Nov 5, 2013

 Personally i'm ok with pip install --upgrade pip printing a message to use python -m pip install --upgrade pip. I just don't think we should throw an error about open file handles or whatever the exception is.
Member

### pfmoore commented Nov 5, 2013

 I was just about to say the same thing. Here's a question - would you want that message to only appear on Windows, or would it be reasonable to say the same on Unix (for a consistent user experience). Personally, I'd like to be consistent across platforms (because it's easier to document that way). Also, we should trap any command that attempts to (re-)install pip - pip install -I pip is another one, for example.
Member

### dstufft commented Nov 5, 2013

 Personally I think it should be windows only. I view it as a work around for the way the Windows platform works, and something that's a target to eventually remove for something nicer down the road.
Member

### pfmoore commented Nov 5, 2013

 I knew you were going to say that :-( Frankly, I don't believe we will ever get "something nicer", basically because we don't have enough Windows expertise to develop and maintain it. I know of no Windows software with a satisfactory self-replacement of executables solution. I certainly don't have an answer, and I wouldn't expect any of the Unix-based developers to be able to help. Even if we did get something, it's likely to be complex and so a probable source of bit rot and obscure bugs. Making a solution that works on all platforms the "official answer" seems to me to avoid sending confusing messages to the user. Back it up with a warning as described and we have a robust solution, even if it's not quite as perfect as getting pip install -U pip to work. Please excuse a certain level of mild grumpiness in this reply, caused by a feeling that it's always the Windows platform that has to be the one to compromise - I don't mean to make an issue out of this. In practice, I'll go with whatever the consensus is. I don't think it's a big deal.
Member

### dstufft commented Nov 5, 2013

 FWIW my goal after PEP453 is done is to get our own testing infrastructure stood up which includes Windows, so hopefully that'll solve some of the windows bitrot problems. I can understand the grumpyness though :) If we were designing this system brand new I'd be a lot more amendable to that but I don't think it's user friendly to break pip install -U pip for everything non Windows in order to maintain consistency. If this was a brand new thing then there's no (or less at least) user epectation :]
Member

### pfmoore commented Nov 5, 2013

 OK, that sounds fair. You're right that historical usage is more important here. I actually have no feel at all for what the user experience on Unix is, so I'm probably just as guilty of one-sided thinking as anyone else. I must try to see how using Python on Linux feels - is there a good "Python on Linux for beginners" resource that shows how to set up a Python (3) environment, get the various tools set up, etc? [One thing I always find doesn't get covered, is how to make "python" run Python 3 by default, how to make "virtualenv" build Python 3 virtualenvs by default, all that sort of thing - it seems to me that Python 3 is much more second-class on Unix than on Windows, oddly enough.]
Member

### dstufft commented Nov 5, 2013

 Typically you don't want to make python point to python3 on *nix because the system uses Python and not all of their libraries are going to be ported to Python3. However there are some Linux distros that have Python3 as a default now... I think Arch? And Maybe the next Fedora?
Member

### pfmoore commented Nov 5, 2013

 Hmm, I'm not talking at a system level, but really you can't choose what to use for Python in your personal account? That definitely makes Python 3 second-class, to my way of thinking... But it does explain why Unix users don't seem to find using things like "pip3" or "pip3.3" as ugly as sin :-) (Anyway, we're way off-topic here, so never mind).
Member

### dstufft commented Nov 5, 2013

 You can setup your shell so it has ~/bin/ before the system bin and add a symlink there from python to python3, but yea offtopic. I'm happy to help though if you're actually interested in getting things set up :] IRC or Email.
Contributor

### vsajip commented Nov 5, 2013

 My work which I believe Paul was referring to was just to implement the 'deleteme' dance for replacing files in use (i.e. exe wrappers) - that's in distlib Python code, not in the .exe wrappers. This is in the distlib.scripts module and should be exercised whenever that module is used to create a script.
Member

### dstufft commented Nov 5, 2013

 So perhaps this is already solved for Wheels? And it's a setuptools problem elsewise?
Member

### pfmoore commented Nov 6, 2013

 Sounds possible. I'll do some tests today, if I can.
Member

Closed

### JingY0604 commented Feb 6, 2017

 How to Solve This Problem?

### piotr-dobrogost commented Feb 6, 2017 • edited

 @JingY0604 See To copy text from a command prompt window for an explanation on how to copy text here. This way the text you post will be searchable.

Merged

Member

### pradyunsg commented Jul 6, 2017 • edited

 @JingY0604 See #3463. You can upgrade your pip by uninstalling the existing installation and using get-pip.py to install the newer version.
Contributor

### ofek commented Sep 8, 2017 • edited

 For https://github.com/ofek/hatch I just catch all instances of updating self and disallow it. I then have an explicit --self flag to do so that Popens a python -m pip install -U hatch and exits immediately. https://github.com/ofek/hatch/blob/22e621715fbed36c2bfab35ae9fa9649ea6dfe1e/hatch/cli.py#L509 edit: technically, instead of just python, sys.executable is used actually
Member

### pradyunsg commented Sep 8, 2017

 @ofek #4490 made a change in pip, printing a better message, IIRC.

### bamarni added a commit to bamarni/dcos-cli that referenced this issue Apr 3, 2018

 Use pypa/pip#1299 on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 d2db203 

### bamarni added a commit to bamarni/dcos-cli that referenced this issue Apr 3, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 d976f9b 

Merged

### bamarni added a commit to dcos/dcos-cli that referenced this issue Apr 4, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 7150fd3 

### bamarni added a commit to bamarni/dcos-cli that referenced this issue Apr 4, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 26dfff1 

### bamarni added a commit to dcos/dcos-cli that referenced this issue Apr 6, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 0ce2110 

### bamarni added a commit to dcos/dcos-cli that referenced this issue Apr 10, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 402bea7 

### bamarni added a commit to dcos/dcos-cli that referenced this issue Apr 10, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 e3955ff 

### ihormelnyk commented Apr 17, 2018

 So for now if I've installed python without pip I will not be able to install pip module in future? c:\Python27>python.exe get-pip.py ERROR: To modify pip, please run the following command: c:\Python27\python.exe -m pip c:\Python27>c:\Python27\python.exe -m pip install c:\Python27\python.exe: No module named pip
Member

### pradyunsg commented Apr 17, 2018

 Hey @ihormelnyk! That's not the case. The conditional for this was too broad -- it blocks using any script with the word "pip" in it from modifying pip. You can try renaming get-pip.py to not contain the word "pip", to work around this issue. A 10.0.1 release (which will likely happen this week) will fix this issue.

### AlexArcPy commented Apr 18, 2018 • edited

 @pradyunsg thanks for sharing the tip. I had the same issue as @ihormelnyk and renaming the get-pip.py file to a name that does not contain the word pip helped.

Merged

### BJQ98 commented Apr 19, 2018

 I print pip in cmd inwindows,but it showed Fatal error in launcher: Unable to create process using '""d:\program files (x86)\python3.6\python.exe" "D:\Program Files (x86)\python3.6\Scripts\pip.exe" ' I have already set environment for pip,but it doesn't work,how to figure it out?
Member

### pradyunsg commented Apr 19, 2018

 @BJQ98 You'd want to look at #5223.

### whtsky added a commit to whtsky/bencoder.pyx that referenced this issue May 1, 2018

 ci(appveyor): upgrade pip using python -m pip install -U pip 
pypa/pip#1299
 cd5035c 

Closed

### supermansaga commented May 7, 2018

 pradyunsg said 10.0.1 would fix this, but i don't know how. Thanks. C:\Users\clarkk\AppData\Local\Programs\Python\Python36>python -m pip install -U pip Collecting pip Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl Installing collected packages: pip Found existing installation: pip 9.0.3 Uninstalling pip-9.0.3: Successfully uninstalled pip-9.0.3 Rolling back uninstall of pip Exception: Traceback (most recent call last): File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\basecommand.py", line 215, in main status = self.run(options, args) File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\commands\install.py", line 342, in run prefix=options.prefix_path, File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\req\req_set.py", line 784, in install **kwargs File "C:\Users\yangg\AppData\Roaming\Python\Python36\site-packages\pip\req\req_install.py", line 851, in install self.move_wheel_files(self.source_dir, root=root, prefix=prefix) File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\req\req_install.py", line 1064, in move_wheel_files isolated=self.isolated, File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip\wheel.py", line 462, in move_wheel_files generated.extend(maker.make(spec)) File "C:\Users\yangg\AppData\Roaming\Python\Python36\site-packages\pip_vendor\distlib\scripts.py", line 372, in make self._make_script(entry, filenames, options=options) File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip_vendor\distlib\scripts.py", line 276, in _make_script self._write_script(scriptnames, shebang, script, filenames, ext) File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip_vendor\distlib\scripts.py", line 212, in _write_script launcher = self._get_launcher('t') File "C:\Users\clarkk\AppData\Roaming\Python\Python36\site-packages\pip_vendor\distlib\scripts.py", line 351, in _get_launcher result = finder(distlib_package).find(name).bytes AttributeError: 'NoneType' object has no attribute 'bytes' You are using pip version 9.0.3, however version 10.0.1 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Member

### pradyunsg commented May 7, 2018

 That'll be the same as #5343.

### supermansaga commented May 7, 2018

 Thanks pradyunsg, Someone else's solution turns out to be simpler. Just performed "python -m pip uninstall pip" first, then "python -m pip install -U pip" Requirement already up-to-date: pip in c:\user\clarkk\appdata\local\Programs\Python\Python36\lib\site-packages (10.0.1)

### bamarni added a commit to dcos/dcos-core-cli that referenced this issue Jul 11, 2018

 Use python -m pip install -U pip on Windows 
Don't run pip install --upgrade pip on Windows, it started to fail in
our Jenkins node and this is not the right command.

https://jira.mesosphere.com/browse/DCOS_OSS-2332
pypa/pip#1299
 46dcefe