Don't fail on missing pip/certifi's cacert.pem

[hroncok]

See https://github.com/pypa/pip/blob/master/src/pip/_vendor/README.rst

It is acceptable to remove the pip/_vendor/requests/cacert.pem file from pip

Virtualenv should not error on that

[gaborbernat]

Let's add some tests for this. You can create e.g. a venv, remove it this file and see that we still pass.

[hroncok]

Sure, tests are great.

What needs to be done in tests:

• create a new virtual environment
• install virtualenv package into it
• patch the pip whl from the virtualenv_support package not to have and use certifi's cacert.pem but instead use /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
• create a new virtual environment using that virtualenv

this currently blows with:

(__venv__) [virtualenvtest]$ python -m virtualenv test_virtualenv
Using base prefix '/usr'
New python executable in /home/churchyard/tmp/virtualenvtest/test_virtualenv/bin/python
Installing setuptools, pip, wheel...

  Complete output from command /home/churchyard/tmp...irtualenv/bin/python - setuptools pip wheel:
  Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
  File "/usr/lib64/python3.7/pkgutil.py", line 637, in get_data
    return loader.get_data(resource_name)
OSError: [Errno 0] Error: 'pip/_vendor/certifi/cacert.pem'
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/churchyard/tmp/virtualenvtest/__venv__/lib64/python3.7/site-packages/virtualenv.py", line 2462, in <module>
    main()
  File "/home/churchyard/tmp/virtualenvtest/__venv__/lib64/python3.7/site-packages/virtualenv.py", line 762, in main
    symlink=options.symlink,
  File "/home/churchyard/tmp/virtualenvtest/__venv__/lib64/python3.7/site-packages/virtualenv.py", line 1015, in create_environment
    install_wheel(to_install, py_executable, search_dirs, download=download)
  File "/home/churchyard/tmp/virtualenvtest/__venv__/lib64/python3.7/site-packages/virtualenv.py", line 968, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "/home/churchyard/tmp/virtualenvtest/__venv__/lib64/python3.7/site-packages/virtualenv.py", line 854, in call_subprocess
    raise OSError("Command {} failed with error code {}".format(cmd_desc, proc.returncode))
OSError: Command /home/churchyard/tmp...irtualenv/bin/python - setuptools pip wheel failed with error code 1

With my patch, it should not.

Now if I provide Python code that does and verifies the above, would you be able to guide me on where to stick that into the current test suite?

[gaborbernat]

Put it at the end of https://github.com/pypa/virtualenv/blob/master/tests/test_virtualenv.py#L334; I would also caution you to not hardcode to etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem, but instead e.g. move pips into a temp folder and point to that. This way the test can succeed independent if etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem exists or not.

[hroncok]

Added a test, verified that it fails without that fix.

[hroncok]

OK, I have no idea what's the problem on Windows now or how to see any details 😢

Help me please?

[gaborbernat]

Paths on windows are using the \ separator instead of /. Use path.join to make your code OS agnostic (path.join(a, "src", "virtualenv_support")).

[hroncok]

Windows still failing even with os.path.join(proj_dir, "src", "virtualenv_support"), no idea why. Also only on Python 3.

https://dev.azure.com/pypa/virtualenv/_build/results?buildId=3012&view=logs&jobId=debe4b1e-a6de-59f5-07ab-92771d5f9a9b&taskId=3601f2ed-d83f-5185-4e8b-01cc25e9c135&lineStart=42&lineEnd=42&colStart=38&colEnd=108

[gaborbernat]

Once we get the tests passing I would like to ask you to break up into a failing test only commit, and then another on top that does the source change and fixes it.

[hroncok]

Once we get the tests passing I would like to ask you to break up into a failing test only commit, and then another on top that does the source change and fixes it.

Shall I mark the test xfail in the first commit or not?

---

I have figured out that if the script I've changed fails, there is no error output anywhere in the test log, so I've temporarily enabled that output for now to see what's wrong on the Windows Python 3 code (maybe it's juts a different exception type). However the CI won't start.

[gaborbernat]

No need for xfail just let it fail. if you ammend your commit it my trigger a redirect.

[gaborbernat]

https://dev.azure.com/pypa/virtualenv/_build/results?buildId=3021

[gaborbernat]

thanks

[hroncok]

Thanks to you.

[gaborbernat]

cool