install_certificates.command too complicated

[dimpase]

BPO	36344
Nosy	@rhettinger, @ronaldoussoren, @benjaminp, @ned-deily, @ambv, @dimpase, @sarnold

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = None
created_at = <Date 2019-03-18.12:24:20.337>
labels = ['OS-mac', '3.8', 'type-feature', '3.7', '3.9']
title = 'install_certificates.command too complicated'
updated_at = <Date 2021-01-19.21:07:03.432>
user = 'https://github.com/dimpase'

bugs.python.org fields:

activity = <Date 2021-01-19.21:07:03.432>
actor = 'ned.deily'
assignee = 'ned.deily'
closed = False
closed_date = None
closer = None
components = ['macOS']
creation = <Date 2019-03-18.12:24:20.337>
creator = 'dimpase'
dependencies = []
files = []
hgrepos = []
issue_num = 36344
keywords = []
message_count = 9.0
messages = ['338211', '338312', '338325', '338752', '338771', '350296', '350516', '385292', '385294']
nosy_count = 7.0
nosy_names = ['rhettinger', 'ronaldoussoren', 'benjamin.peterson', 'ned.deily', 'lukasz.langa', 'dimpase', 'sarnold']
pr_nums = []
priority = 'critical'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue36344'
versions = ['Python 2.7', 'Python 3.7', 'Python 3.8', 'Python 3.9']

[dimpase]

Currently (e.g. on the released Python 2.7.16) Mac/BuildScript/resources/install_certificates.command does install certifi module from the net and symlinks its cacert.pem to provide openssl with a working certificate. The same task may be accomplished much easier, by symlinking pip's cacert.pem, as follows (just shell commands, for the purposes of demonstration)

cd local/openssl
rm -f local/openssl/cert.pem
ln -s ../lib/python2.7/site-packages/pip/_vendor/certifi/cacert.pem cert.pem

This works as pip's cacert.pem contains the same certificate as the one provided by unvendored certifi (as can be seen by looking at it using "openssl x509 -in ..." on it).

I'd be happy to provide a PR if this is acceptable.

[ned-deily]

Thanks for the suggestion but that is not a workable solution for two reasons. One, pip is an optional install with the python.org installer so we cannot depend on it being available. More importantly, from a packaging point of view, the internal composition of pip is opaque. There's no guarantee that any future release of pip will still bundle root certificates, that they will be installed in the same location, or which root certificates will be included and how up-to-date they are. Pip uses the certificates primarily to access PyPI, not to provide a general set of root certificates. The current python.org solution of providing the Install Certificates script as an example is certainly far from ideal and we will improve it. But adding a dependency on undocumented behavior of pip is not a step in the right direction.

[dimpase]

The script install_certificates.command depends upon pip, it calls pip to install certifi. Thus it's no less "optional" than pip.

And pip is only functional, and it able to do the installation in question, due to it including the certificate in question.

The role of this script is fishy from security point of view, too.
Why not simply putting the certificate right where it belongs to, i.e. not just simplify install_certificates.command, but simply get rid of it?

[ned-deily]

I do not disagree that the current manual Install Certificates step is not ideal but, again, for the reasons cited in my earlier response (and other reasons), adding a dependency on pip to provide certificates is not a good idea. But, since there does not seem to be another open issue about this right now, I am going to reopen this one and use it to implement a solution that eliminates the need to manually run Install Certificates at installation time.

[rhettinger]

I am going to reopen this one and use it to implement a solution
that eliminates the need to manually run Install Certificates
at installation time.

There will be much rejoicing. Almost every week, I have a learner bump into this issue.

[ambv]

This is marked as a release blocker. The last 3.8 beta is scheduled for Monday. Please decide how to proceed ASAP.

[ned-deily]

So as not to delay 3.8.0b4, I'm removing this as a "release blocker'. Once the implementation is in master, we can consider backporting to other releases.

[sarnold]

I can't seem to find a usable answer for a user on Sierra with a (supposedly) working python 3.6 install. From what I can tell all the related bugs are closed except this one, and this one doesn't include python 3.6. Running the install_certificates.command does not work in his case and double-clicking results in a permissions error. I suspect the target system is not in the best shape, but I haven't touched a Mac since the powerpc days.

What exactly is the backport status for this issue, and is there anything with more info for this user?

[ned-deily]

What exactly is the backport status for this issue, and is there anything with more info for this user?

Python 3.6 has been in the security-fix-only phase of its life cycle since 2018-12 which means that python.org binary installers for Mac or Windows platforms have not been produced since then. If at all possible, your user should upgrade to a fully-supported version, currently either Python 3.9.1 or 3.8.7 (https://www.python.org/downloads/). That said, the last python.org macOS installers for 3.6, 3.6.8, still work for me on the most recent macOS Sierra update (10.12.6) including the "Install Certificates Command". Make sure the user has admin privileges or that "Install Certificates Commnad" is run under a user that does have admin privs. If it still fails, you could try downloading and re-installing Python 3.6.8 or give more information about exactly what error is being seen. But this sort of problem is really off-topic for this issue; better to ask in a help forum.