Invalid version reported by conda list / conda env export when replacing pip package with conda

[beenje]

What happened?

I created a conda env with some pip packages. I then updated that env to replace the pip package with a more recent conda one. The package seems to be updated but conda list and conda env export still show the old pip package being installed.

Here is to reproduce the issue:

1. Create environment

$ cat environment-pip.yml
name: foo
dependencies:
  - python=3.9
  - pip
  - pip:
      - requests==2.26.0

$ conda env create -f environment-pip.yml
$ conda list -n foo
# packages in environment at /opt/conda/envs/foo:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
certifi                   2021.10.8                pypi_0    pypi
charset-normalizer        2.0.12                   pypi_0    pypi
idna                      3.3                      pypi_0    pypi
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_16    conda-forge
libgomp                   11.2.0              h1d223b6_16    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.11            h166bdaf_1014    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
openssl                   3.0.2                h166bdaf_1    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
python                    3.9.12          h2660328_1_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
requests                  2.26.0                   pypi_0    pypi
setuptools                62.1.0           py39hf3d152e_0    conda-forge
sqlite                    3.38.3               h4ff8645_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
urllib3                   1.26.9                   pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h166bdaf_1014    conda-forge

$ grep version /opt/conda/envs/foo/lib/python3.9/site-packages/requests/__version__.py
__version__ = '2.26.0'
$ ls -ld /opt/conda/envs/foo/lib/python3.9/site-packages/requests*
drwxr-xr-x 3 root root 4096 May  3 12:34 /opt/conda/envs/foo/lib/python3.9/site-packages/requests
drwxr-xr-x 2 root root 4096 May  3 12:34 /opt/conda/envs/foo/lib/python3.9/site-packages/requests-2.26.0.dist-info

2. Update environment

$ cat environment-conda.yml
name: foo
dependencies:
  - python=3.9
  - requests=2.27.1

$ conda env update -f environment-conda.yml
$ conda list -n foo
# packages in environment at /opt/conda/envs/foo:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
brotlipy                  0.7.0           py39hb9d737c_1004    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
certifi                   2021.10.8                pypi_0    pypi
cffi                      1.15.0           py39h4bc2ebd_0    conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
cryptography              36.0.2           py39hd97740a_1    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_16    conda-forge
libgomp                   11.2.0              h1d223b6_16    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.11            h166bdaf_1014    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
openssl                   1.1.1n               h166bdaf_0    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1            py39hf3d152e_5    conda-forge
python                    3.9.12          h9a8a25e_1_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
requests                  2.26.0                   pypi_0    pypi
setuptools                62.1.0           py39hf3d152e_0    conda-forge
sqlite                    3.38.3               h4ff8645_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
urllib3                   1.26.9             pyhd8ed1ab_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h166bdaf_1014    conda-forge

$ conda env export -n foo
name: foo
channels:
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - brotlipy=0.7.0=py39hb9d737c_1004
  - bzip2=1.0.8=h7f98852_4
  - ca-certificates=2021.10.8=ha878542_0
  - cffi=1.15.0=py39h4bc2ebd_0
  - charset-normalizer=2.0.12=pyhd8ed1ab_0
  - cryptography=36.0.2=py39hd97740a_1
  - idna=3.3=pyhd8ed1ab_0
  - ld_impl_linux-64=2.36.1=hea4e1c9_2
  - libffi=3.4.2=h7f98852_5
  - libgcc-ng=11.2.0=h1d223b6_16
  - libgomp=11.2.0=h1d223b6_16
  - libnsl=2.0.0=h7f98852_0
  - libuuid=2.32.1=h7f98852_1000
  - libzlib=1.2.11=h166bdaf_1014
  - ncurses=6.3=h27087fc_1
  - openssl=1.1.1n=h166bdaf_0
  - pip=22.0.4=pyhd8ed1ab_0
  - pycparser=2.21=pyhd8ed1ab_0
  - pyopenssl=22.0.0=pyhd8ed1ab_0
  - pysocks=1.7.1=py39hf3d152e_5
  - python=3.9.12=h9a8a25e_1_cpython
  - python_abi=3.9=2_cp39
  - readline=8.1=h46c0cb4_0
  - setuptools=62.1.0=py39hf3d152e_0
  - sqlite=3.38.3=h4ff8645_0
  - tk=8.6.12=h27826a3_0
  - tzdata=2022a=h191b570_0
  - urllib3=1.26.9=pyhd8ed1ab_0
  - wheel=0.37.1=pyhd8ed1ab_0
  - xz=5.2.5=h516909a_1
  - zlib=1.2.11=h166bdaf_1014
  - pip:
    - certifi==2021.10.8
    - requests==2.26.0

$ ls -ld /opt/conda/envs/foo/lib/python3.9/site-packages/requests*
drwxr-xr-x 3 root root 4096 May  3 12:43 /opt/conda/envs/foo/lib/python3.9/site-packages/requests
drwxr-xr-x 2 root root 4096 May  3 12:34 /opt/conda/envs/foo/lib/python3.9/site-packages/requests-2.26.0.dist-info
drwxr-xr-x 2 root root 4096 May  3 12:43 /opt/conda/envs/foo/lib/python3.9/site-packages/requests-2.27.1.dist-info

$ grep version /opt/conda/envs/foo/lib/python3.9/site-packages/requests/__version__.py
__version__ = '2.27.1'

requests 2.27.1 was properly installed but conda list and conda env export still reports the version 2.26.0 being installed with pip. I guess it's because /opt/conda/envs/foo/lib/python3.9/site-packages/requests-2.26.0.dist-info wasn't deleted.

Conda Details

conda info

active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files : /opt/conda/.condarc
          conda version : 4.12.0
    conda-build version : 3.21.8
         python version : 3.9.10.final.0
       virtual packages : __linux=5.10.76=0
                          __glibc=2.17=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.9.10 Linux/5.10.76-linuxkit centos/7.9.2009 glibc/2.17
                UID:GID : 0:0
             netrc file : None
           offline mode : False

conda config

==> /opt/conda/.condarc <==
auto_update_conda: False
channels:
  - conda-forge
show_channel_urls: True

conda list

# packages in environment at /opt/conda/envs/foo:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
brotlipy                  0.7.0           py39hb9d737c_1004    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
certifi                   2021.10.8                pypi_0    pypi
cffi                      1.15.0           py39h4bc2ebd_0    conda-forge
charset-normalizer        2.0.12             pyhd8ed1ab_0    conda-forge
cryptography              36.0.2           py39hd97740a_1    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 11.2.0              h1d223b6_16    conda-forge
libgomp                   11.2.0              h1d223b6_16    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.11            h166bdaf_1014    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
openssl                   1.1.1n               h166bdaf_0    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1            py39hf3d152e_5    conda-forge
python                    3.9.12          h9a8a25e_1_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
readline                  8.1                  h46c0cb4_0    conda-forge
requests                  2.26.0                   pypi_0    pypi
setuptools                62.1.0           py39hf3d152e_0    conda-forge
sqlite                    3.38.3               h4ff8645_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
urllib3                   1.26.9             pyhd8ed1ab_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h166bdaf_1014    conda-forge

[kenodegard]

@beenje It's because pip uninstall requests isn't called, so as you noted, the dist-info from pip's install for requests 2.26 is still present. Python correctly uses the conda installed requests 2.27, so this appears to only be a metadata issue. The manual fix would be to either remove the old dist-infos or to run a pip uninstall before a conda install (obviously, neither is desired, so this is still something that needs to be addressed).

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

[beenje]

The issue is still reproducible with conda 23.5.2:

conda info

     active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files : /opt/conda/.condarc
          conda version : 23.5.2
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.31=0
                          __linux=5.15.49=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/23.5.2 requests/2.31.0 CPython/3.10.12 Linux/5.15.49-linuxkit ubuntu/20.04.6 glibc/2.31
                UID:GID : 0:0
             netrc file : None
           offline mode : False

[LFKoning]

Just tagging this, because I had the exact same issue. Installed a package using pip first, then reinstalled using conda. The Python interpreter correctly uses the conda version, but conda list shows the old pip version (same as pip list btw). Using conda version 24.3.0.