Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conda doesn't correctly uninstalls packages installed by pip #10357

Closed
stas00 opened this issue Nov 13, 2020 · 4 comments
Closed

conda doesn't correctly uninstalls packages installed by pip #10357

stas00 opened this issue Nov 13, 2020 · 4 comments
Labels
locked [bot] locked due to inactivity stale::closed [bot] closed after being marked as stale stale [bot] marked as stale due to inactivity

Comments

@stas00
Copy link

stas00 commented Nov 13, 2020

Current Behavior

conda does not remove all files listed in the RECORD metadata file when asked to remove a package. This leaves a corrupt distribution present, which other tools such as pip are unable to process.

Steps to Reproduce

Let's pick a package pynvml as an example:

# cleanup
conda uninstall -y pynvml
pip uninstall -y pynvml
# must install/uninstall things in this exact order
conda install -y -c conda-forge pynvml
pip uninstall -y pynvml
pip install pynvml
conda uninstall -y pynvml
# check: not good:
ls -lt /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages/pynvml-8.0.4.dist-info/
total 0
-rw-rw-r-- 1 stas stas 0 Nov 12 22:03 REQUESTED

conda didn't delete the file listed in RECORD

This file wasn't in conda-forge build, but was added by pip - it seems conda removes only files that it installed,

So now the site-packages is broken and we can't use pip anymore:

pip install pynvml
Requirement already satisfied: pynvml in /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages (8.0.4)
WARNING: No metadata found in /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages
ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages/pynvml-8.0.4.dist-info/METADATA'

"Requirement already satisfied" is bogus since there is just one REQUESTED file that wasn't cleaned up in the dist dir.

this fails too:

pip uninstall -y pynvml
Found existing installation: pynvml 8.0.4
ERROR: Exception:
Traceback (most recent call last):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/commands/uninstall.py", line 89, in run
    uninstall_pathset = req.uninstall(
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 685, in uninstall
    uninstalled_pathset = UninstallPathSet.from_dist(dist)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py", line 535, in from_dist
    for path in uninstallation_paths(dist):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py", line 67, in unique
    for item in fn(*args, **kw):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py", line 85, in uninstallation_paths
    r = csv.reader(FakeFile(dist.get_metadata_lines('RECORD')))
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1432, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1420, in get_metadata
    value = self._get(path)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1616, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages/pynvml-8.0.4.dist-info/RECORD'

The only fix at the moment is to manually remove the dir by hand.

I have a package whose build process validates that the conda/pip installs worked correctly and so it happens to hit a similar sequence. But this can easily happen in real life, except install / uninstall commands can be called days apart...

Expected Behavior

conda and pip need to find a way to work better together ;)

I initially thought that it was a bug in pip, and filed a bug report there
pypa/pip#9129
but pip developers indicated that it's a bug in conda instead.

Thank you!

Environment Information

`conda info`

     active environment : main-38
    active env location : /home/stas/anaconda3/envs/main-38
            shell level : 5
       user config file : /home/stas/.condarc
 populated config files :
          conda version : 4.9.2
    conda-build version : 3.18.9
         python version : 3.7.4.final.0
       virtual packages : __cuda=11.1=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/stas/anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/stas/anaconda3/pkgs
                          /home/stas/.conda/pkgs
       envs directories : /home/stas/anaconda3/envs
                          /home/stas/.conda/envs
               platform : linux-64
             user-agent : conda/4.9.2 requests/2.24.0 CPython/3.7.4 Linux/5.4.0-52-generic ubuntu/20.04.1 glibc/2.31
                UID:GID : 1000:1000
             netrc file : /home/stas/.netrc
           offline mode : False

  • pip version: 20.2.4
@pradyunsg
Copy link

FYI -- we've been getting reports about failures like this since support for the REQUESTED file (from PEP 376) was added into pip.

It was only when OP filed an issue detailing their entire sequence of reproduction steps (cross-linked by GitHub above) that we got to know what was likely the root cause of that issue. :)

@stas00
Copy link
Author

stas00 commented Nov 14, 2020

It took quite some trial-n-error to figure out that sequence. Thank you for the appreciation note, @pradyunsg.

@github-actions
Copy link

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

  1. Verify that you can still reproduce the issue at hand
  2. Comment that the issue is still reproducible and include:
    - What OS and version you reproduced the issue on
    - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

@github-actions github-actions bot added the stale [bot] marked as stale due to inactivity label Jan 22, 2023
@github-project-automation github-project-automation bot moved this to 🆕 New in 🧭 Planning Mar 22, 2023
@github-actions github-actions bot added the stale::closed [bot] closed after being marked as stale label Mar 24, 2023
@github-project-automation github-project-automation bot moved this from 🆕 New to 🏁 Done in 🧭 Planning Mar 24, 2023
@buhrmann
Copy link

Hi, just wondering if anybody has looked at solving this? Still seems to be a problem...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
locked [bot] locked due to inactivity stale::closed [bot] closed after being marked as stale stale [bot] marked as stale due to inactivity
Projects
Archived in project
Development

No branches or pull requests

3 participants