pip freeze on a fresh Miniconda env returns certifi @ file:///opt/conda/conda-bld/certifi_1655968806487/work/certifi

[joshuacwnewton]

Checklist

• I added a descriptive title
• I searched for other issues and couldn't find a solution or duplication
• I already searched in Google and didn't find any good information or help
• I looked at the docs and didn't see anything to help

What happened?

On June 22nd, 2022, our GitHub Actions ubuntu-latest CI runs were working as usual.

Then, on June 23rd, 2022, our CI workflow runs started failing with the following error:

Processing /opt/conda/conda-bld/certifi_1655968806487/work/certifi
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/opt/conda/conda-bld/certifi_1655968806487/work/certifi'

I traced the source of the error to the following (reproducible) steps:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -p ./test-miniconda3 -b -f
test-miniconda3/bin/conda create -y -n test-env python=3.7
./test-miniconda3/envs/test-env/bin/pip freeze

On my local Ubuntu 22.04 machine, running pip freeze prints:

joshua@tadpole:~$ ./test-miniconda3/envs/test-env/bin/pip freeze
certifi @ file:///opt/conda/conda-bld/certifi_1655968806487/work/certifi

I believe this kind of output is what caused our CI error, since in our use-case, we pip freeze into a requirements.txt file, then try to pip install -r requirements.txt. But, we can't pip install any @ file:// entries, as they don't exist. 😝

Conda info

active environment : None
       user config file : /home/joshua/.condarc
 populated config files : /home/joshua/.condarc
          conda version : 4.12.0
    conda-build version : not installed
         python version : 3.9.12.final.0
       virtual packages : __linux=5.17.5=0
                          __glibc=2.35=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/joshua/test-miniconda3  (writable)
      conda av data dir : /home/joshua/test-miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/joshua/test-miniconda3/pkgs
                          /home/joshua/.conda/pkgs
       envs directories : /home/joshua/test-miniconda3/envs
                          /home/joshua/.conda/envs
               platform : linux-64
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.9.12 Linux/5.17.5-76051705-generic pop/22.04 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

Conda config

==> /home/joshua/.condarc <==
auto_activate_base: False

Conda list

# packages in environment at /home/joshua/test-miniconda3:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
_openmp_mutex             4.5                       1_gnu    defaults
brotlipy                  0.7.0           py39h27cfd23_1003    defaults
ca-certificates           2022.3.29            h06a4308_1    defaults
certifi                   2021.10.8        py39h06a4308_2    defaults
cffi                      1.15.0           py39hd667e15_1    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
colorama                  0.4.4              pyhd3eb1b0_0    defaults
conda                     4.12.0           py39h06a4308_0    defaults
conda-content-trust       0.1.1              pyhd3eb1b0_0    defaults
conda-package-handling    1.8.1            py39h7f8727e_0    defaults
cryptography              36.0.0           py39h9ce1e76_0    defaults
idna                      3.3                pyhd3eb1b0_0    defaults
ld_impl_linux-64          2.35.1               h7274673_9    defaults
libffi                    3.3                  he6710b0_2    defaults
libgcc-ng                 9.3.0               h5101ec6_17    defaults
libgomp                   9.3.0               h5101ec6_17    defaults
libstdcxx-ng              9.3.0               hd4cf53a_17    defaults
ncurses                   6.3                  h7f8727e_2    defaults
openssl                   1.1.1n               h7f8727e_0    defaults
pip                       21.2.4           py39h06a4308_0    defaults
pycosat                   0.6.3            py39h27cfd23_0    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pyopenssl                 22.0.0             pyhd3eb1b0_0    defaults
pysocks                   1.7.1            py39h06a4308_0    defaults
python                    3.9.12               h12debd9_0    defaults
readline                  8.1.2                h7f8727e_1    defaults
requests                  2.27.1             pyhd3eb1b0_0    defaults
ruamel_yaml               0.15.100         py39h27cfd23_0    defaults
setuptools                61.2.0           py39h06a4308_0    defaults
six                       1.16.0             pyhd3eb1b0_1    defaults
sqlite                    3.38.2               hc218d9a_0    defaults
tk                        8.6.11               h1ccaba5_0    defaults
tqdm                      4.63.0             pyhd3eb1b0_0    defaults
tzdata                    2022a                hda174b7_0    defaults
urllib3                   1.26.8             pyhd3eb1b0_0    defaults
wheel                     0.37.1             pyhd3eb1b0_0    defaults
xz                        5.2.5                h7b6447c_0    defaults
yaml                      0.2.5                h7b6447c_0    defaults
zlib                      1.2.12               h7f8727e_1    defaults

Additional Context

• Before June 23rd, I was able to pip freeze and get certifi==2022.6.15, which is the latest version of certifi as per conda-forge/certifi-feedstock@3ded6d9. So, I don't think this is an issue on certifi's end?

• Also, a quick GitHub-wide search for certifi_1655968806487 returns my issue and 1 other result 2 10 36 other results (so far).

  The requirements.txt file from the first result contains many @ file:// entries, leading me to believe that this issue is a conda-specific issue that affects more than just the certifi package.

[joshuacwnewton]

Right now, I'm running pip freeze | grep -v "@ file" > requirements.txt to temporarily work around this issue, since in our use-case, this is only affecting certifi, which we can safely exclude from our requirements.txt, I think.

But, I imagine that won't work for many people who rely on actually capturing the packages that are affected by this.

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Getting this as well. On Mac, getting:

certifi @ file:///private/var/folders/sy/f16zz6x50xz3113nwtb9bvq00000gp/T/abs_83242e7e-f82d-4a71-8ef2-9d71d212d249gu_wxmeq/croots/recipe/certifi_1655968827803/work/certifi

I'm personally very surprised by this, since we pin everything; does miniconda automatically phone home to update + reinstall packages? (maybe this is related to aggressive_update_packages?) Doing some grepping, I'm finding a couple things:

1. I see the same path appear in miniconda-py39_4.11.0/pkgs/certifi-2022.6.15-py37hecd8cb5_0/info/recipe/meta.yaml
2. The only other place it's appearing is in miniconda-py39_4.11.0/pkgs/certifi-2022.6.15-py37hecd8cb5_0/lib/python3.7/site-packages/certifi-2022.6.15.dist-info/direct_url.json, which is probably where pip freeze is getting its info

I can also workaround, but this seems like a major issue that should be fixed ASAP

[rolweber]

This is not a conda problem. It's the feedstock recipes for various packages interacting poorly with pip. When repackaging PyPI packages as conda packages, many recipes run pip install with a URL, which creates a direct_url.json file in the conda package. This later causes pip to report the file:/// URLs instead of the plain package version.

[joshuacwnewton]

This is not a conda problem. It's the feedstock recipes for various packages interacting poorly with pip.

certifi-feedstock's recipe seemingly hasn't changed in years (beyond automated metadata updates). So, if it's a recipe issue, as you say, then I'm curious why the issue hadn't occurred before June 23rd? 🤔

EDIT: See #11580 (comment).

[botastark]

Encountered the same problem after running pip freeze. Although before that it was installing properly

[rolweber]

The introduction of direct_url.json was a change in pip, and recipes are calling pip.

https://peps.python.org/pep-0610/
https://pip.pypa.io/en/stable/news/#b1-2020-04-21

[rolweber]

For the conda envs I'm creating in container image builds, my workaround is this:

find $CONDA_PREFIX/lib/python*/site-packages/ \
     -maxdepth 2 -name direct_url.json \
     -exec rm -f {} +

[joshuacwnewton]

The introduction of direct_url.json was a change in pip, and recipes are calling pip.

https://peps.python.org/pep-0610/ https://pip.pypa.io/en/stable/news/#b1-2020-04-21

Thank you for the links and additional context.

The reason I'm still a bit confused is that, if direct_url.json was introduced in 2020-04-21, why is it that the certifi issues had only started appearing in 2022-06-23?

[rolweber]

That was a beta version of pip. After that:

1. The feature had to go into a regular release of pip.
2. Anaconda had to update their pip package to that regular release, or a later one.
3. certifi had to get re-built using the updated pip package from Anaconda.

Unfortunately, I don't know how to determine which version of pip was used to build a particular version of certifi.

[jaimergp]

It looks like you are mainly using defaults, so if it's a feedstock issue, then it's this one. And the last commit there was pushed... 22 days ago 🕵️

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!