Pip uninstall a old version failed when we specify a newer version to install

[MrKiven]

• Pip version: 9.0.1
• Python version: 2.7.5
• Operating system: centos 7.1.1503

Description:

we try to install a newer version of a package , pip will uninstall old version first, but uninstall failed sometimes when try to remove a console-scripts.

In setup.py:

entry_points = {
    'console_scripts': [
        'ves = some_path',
    ],

Then the exception is:

Found existing installation: mypackage-x.y.z
Uninstalling mypackage-x.y.z:
Exception:
Traceback (most recent call last):
File "/srv/ves/.vespenenv/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
status = self.run(options, args)
File "/srv/ves/.vespenenv/lib/python2.7/site-packages/pip/commands/install.py", line 311, in run
root=options.root_path,
File "/srv/ves/.vespenenv/lib/python2.7/site-packages/pip/req/req_set.py", line 640, in install
requirement.uninstall(auto_confirm=True)
File "/srv/ves/.vespenenv/lib/python2.7/site-packages/pip/req/req_install.py", line 716, in uninstall
paths_to_remove.remove(auto_confirm)
File "/srv/ves/.vespenenv/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
renames(path, new_path)
File "/srv/ves/.vespenenv/lib/python2.7/site-packages/pip/utils/__init__.py", line 315, in renames
shutil.move(old, new)
File "/usr/lib64/python2.7/shutil.py", line 301, in move
copy2(src, real_dst)
File "/usr/lib64/python2.7/shutil.py", line 130, in copy2
copyfile(src, dst)
File "/usr/lib64/python2.7/shutil.py", line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/srv/ves/.vespenenv/bin/ves'

What I've run:

we just run this in start.sh：

/srv/ves/.vespenenv/bin/pip install mypackage==x.y.z+1 -f .wheelhouse --no-index

This has already perplexed me for a long time, it does not always reproduce...

[xavfernandez]

Related to #3801

[MrKiven]

yeah..we first package the wheels, then pip install xxx -f .wheelhouse --no-index， it's reproducible. Any plan to fix it ?

[chrahunt]

I can't reproduce this with 19.2.1:

repro.sh

#!/bin/sh

cd $(mktemp -d)

python -m venv .venv
. ./.venv/bin/activate

set -x
pip install --upgrade pip wheel
pip --version
mkdir project


cd project/
cat <<EOF > setup.py
from setuptools import setup


setup(
    name='project',
    version='0.1.1',

    packages=['project',],
    entry_points={
        'console_scripts': ['cli = project:main',],
    },
)
EOF
mkdir project
touch project/__init__.py
python setup.py bdist_wheel
cd ..

echo "### Initial install"
diff -u /dev/null project/setup.py
pip install -f ./project/dist --no-index project


cd project
cp setup.py setup.py.bk
sed -i 's/0.1.0/0.1.1/g' setup.py
python setup.py bdist_wheel
cd ..

echo "### Plain upgrade, no change to console_scripts."
diff -u project/setup.py.bk project/setup.py
pip install -f ./project/dist --no-index project==0.1.1


cd project
cp setup.py setup.py.bk
sed -i -e 's/0.1.1/0.1.2/g' -e 's/cli =/foo =/g' setup.py
python setup.py bdist_wheel
cd ..

echo "### Upgrade, remove and add a console script."
diff -u project/setup.py.bk project/setup.py
pip install -f ./project/dist --no-index project==0.1.2


cd project
cp setup.py setup.py.bk
sed -i -e 's/0.1.2/0.1.3/g' -e "s/\\['foo =.*\\]/[]/g" setup.py
python setup.py bdist_wheel
cd ..

echo "### Upgrade, add a console script."
diff -u project/setup.py.bk project/setup.py
pip install -f ./project/dist --no-index project==0.1.3

Output

+ pip install --upgrade pip wheel
Collecting pip
  Using cached https://files.pythonhosted.org/packages/62/ca/94d32a6516ed197a491d17d46595ce58a83cbb2fca280414e57cd86b84dc/pip-19.2.1-py2.py3-none-any.whl
Collecting wheel
  Using cached https://files.pythonhosted.org/packages/bb/10/44230dd6bf3563b8f227dbf344c908d412ad2ff48066476672f3a72e174e/wheel-0.33.4-py2.py3-none-any.whl
Installing collected packages: pip, wheel
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
Successfully installed pip-19.2.1 wheel-0.33.4
+ pip --version
pip 19.2.1 from /tmp/user/1000/tmp.SoJE56BKBs/.venv/lib/python3.7/site-packages/pip (python 3.7)
+ mkdir project
+ cd project/
+ cat
+ mkdir project
+ touch project/__init__.py
+ python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/project
copying project/__init__.py -> build/lib/project
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/project
copying build/lib/project/__init__.py -> build/bdist.linux-x86_64/wheel/project
running install_egg_info
running egg_info
creating project.egg-info
writing project.egg-info/PKG-INFO
writing dependency_links to project.egg-info/dependency_links.txt
writing entry points to project.egg-info/entry_points.txt
writing top-level names to project.egg-info/top_level.txt
writing manifest file 'project.egg-info/SOURCES.txt'
reading manifest file 'project.egg-info/SOURCES.txt'
writing manifest file 'project.egg-info/SOURCES.txt'
Copying project.egg-info to build/bdist.linux-x86_64/wheel/project-0.1.1-py3.7.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/project-0.1.1.dist-info/WHEEL
creating 'dist/project-0.1.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'project/__init__.py'
adding 'project-0.1.1.dist-info/METADATA'
adding 'project-0.1.1.dist-info/WHEEL'
adding 'project-0.1.1.dist-info/entry_points.txt'
adding 'project-0.1.1.dist-info/top_level.txt'
adding 'project-0.1.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ cd ..
+ echo ### Initial install
### Initial install
+ diff -u /dev/null project/setup.py
--- /dev/null   2019-08-09 18:53:01.156613370 -0400
+++ project/setup.py    2019-08-10 11:11:05.149026736 -0400
@@ -0,0 +1,12 @@
+from setuptools import setup
+
+
+setup(
+    name='project',
+    version='0.1.1',
+
+    packages=['project',],
+    entry_points={
+        'console_scripts': ['cli = project:main',],
+    },
+)
+ pip install -f ./project/dist --no-index project
Looking in links: ./project/dist
Collecting project
Installing collected packages: project
Successfully installed project-0.1.1
+ cd project
+ cp setup.py setup.py.bk
+ sed -i s/0.1.0/0.1.1/g setup.py
+ python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/project
copying build/lib/project/__init__.py -> build/bdist.linux-x86_64/wheel/project
running install_egg_info
running egg_info
writing project.egg-info/PKG-INFO
writing dependency_links to project.egg-info/dependency_links.txt
writing entry points to project.egg-info/entry_points.txt
writing top-level names to project.egg-info/top_level.txt
reading manifest file 'project.egg-info/SOURCES.txt'
writing manifest file 'project.egg-info/SOURCES.txt'
Copying project.egg-info to build/bdist.linux-x86_64/wheel/project-0.1.1-py3.7.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/project-0.1.1.dist-info/WHEEL
creating 'dist/project-0.1.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'project/__init__.py'
adding 'project-0.1.1.dist-info/METADATA'
adding 'project-0.1.1.dist-info/WHEEL'
adding 'project-0.1.1.dist-info/entry_points.txt'
adding 'project-0.1.1.dist-info/top_level.txt'
adding 'project-0.1.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ cd ..
+ echo ### Plain upgrade, no change to console_scripts.
### Plain upgrade, no change to console_scripts.
+ diff -u project/setup.py.bk project/setup.py
+ pip install -f ./project/dist --no-index project==0.1.1
Looking in links: ./project/dist
Requirement already satisfied: project==0.1.1 in ./.venv/lib/python3.7/site-packages (0.1.1)
+ cd project
+ cp setup.py setup.py.bk
+ sed -i -e s/0.1.1/0.1.2/g -e s/cli =/foo =/g setup.py
+ python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/project
copying build/lib/project/__init__.py -> build/bdist.linux-x86_64/wheel/project
running install_egg_info
running egg_info
writing project.egg-info/PKG-INFO
writing dependency_links to project.egg-info/dependency_links.txt
writing entry points to project.egg-info/entry_points.txt
writing top-level names to project.egg-info/top_level.txt
reading manifest file 'project.egg-info/SOURCES.txt'
writing manifest file 'project.egg-info/SOURCES.txt'
Copying project.egg-info to build/bdist.linux-x86_64/wheel/project-0.1.2-py3.7.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/project-0.1.2.dist-info/WHEEL
creating 'dist/project-0.1.2-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'project/__init__.py'
adding 'project-0.1.2.dist-info/METADATA'
adding 'project-0.1.2.dist-info/WHEEL'
adding 'project-0.1.2.dist-info/entry_points.txt'
adding 'project-0.1.2.dist-info/top_level.txt'
adding 'project-0.1.2.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ cd ..
+ echo ### Upgrade, remove and add a console script.
### Upgrade, remove and add a console script.
+ diff -u project/setup.py.bk project/setup.py
--- project/setup.py.bk 2019-08-10 11:11:06.145029584 -0400
+++ project/setup.py    2019-08-10 11:11:06.145029584 -0400
@@ -3,10 +3,10 @@

 setup(
     name='project',
-    version='0.1.1',
+    version='0.1.2',

     packages=['project',],
     entry_points={
-        'console_scripts': ['cli = project:main',],
+        'console_scripts': ['foo = project:main',],
     },
 )
+ pip install -f ./project/dist --no-index project==0.1.2
Looking in links: ./project/dist
Collecting project==0.1.2
Installing collected packages: project
  Found existing installation: project 0.1.1
    Uninstalling project-0.1.1:
      Successfully uninstalled project-0.1.1
Successfully installed project-0.1.2
+ cd project
+ cp setup.py setup.py.bk
+ sed -i -e s/0.1.2/0.1.3/g -e s/\['foo =.*\]/[]/g setup.py
+ python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/project
copying build/lib/project/__init__.py -> build/bdist.linux-x86_64/wheel/project
running install_egg_info
running egg_info
writing project.egg-info/PKG-INFO
writing dependency_links to project.egg-info/dependency_links.txt
writing entry points to project.egg-info/entry_points.txt
writing top-level names to project.egg-info/top_level.txt
reading manifest file 'project.egg-info/SOURCES.txt'
writing manifest file 'project.egg-info/SOURCES.txt'
Copying project.egg-info to build/bdist.linux-x86_64/wheel/project-0.1.3-py3.7.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/project-0.1.3.dist-info/WHEEL
creating 'dist/project-0.1.3-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'project/__init__.py'
adding 'project-0.1.3.dist-info/METADATA'
adding 'project-0.1.3.dist-info/WHEEL'
adding 'project-0.1.3.dist-info/entry_points.txt'
adding 'project-0.1.3.dist-info/top_level.txt'
adding 'project-0.1.3.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ cd ..
+ echo ### Upgrade, add a console script.
### Upgrade, add a console script.
+ diff -u project/setup.py.bk project/setup.py
--- project/setup.py.bk 2019-08-10 11:11:06.645031014 -0400
+++ project/setup.py    2019-08-10 11:11:06.645031014 -0400
@@ -3,10 +3,10 @@

 setup(
     name='project',
-    version='0.1.2',
+    version='0.1.3',

     packages=['project',],
     entry_points={
-        'console_scripts': ['foo = project:main',],
+        'console_scripts': [],
     },
 )
+ pip install -f ./project/dist --no-index project==0.1.3
Looking in links: ./project/dist
Collecting project==0.1.3
Installing collected packages: project
  Found existing installation: project 0.1.2
    Uninstalling project-0.1.2:
      Successfully uninstalled project-0.1.2
Successfully installed project-0.1.3

Are we missing anything here that may be relevant?

[chrahunt]

It's been some time without a response, so I'll close this. Please let us know if you're still experiencing any problems. Thanks!