Why are python tools not using the ca-certificates bundle? Can we fix this with softlinks to ca-certificates?

[reegnz]

Output of brew config

None

Output of brew doctor

None

Description of issue

I believe it is good practice for the tools installed by a package manager to use the same trust store across the board.

I'm seeing that homebrew has ca-certificates formula, which is then wired nicely into a bunch of tools, and also takes into consideration certificates installed into the MacOS keychain (as it should).

Now there are several tools written in python that don't respect this currently, and instead of using the ca-certificates bundle, they use some arbitrary pre-packaged certificate.

Example tools I currently am using that are affected:

• checkov - uses certifi cacert.pem by default
• jc - uses certifi cacert.pem by default
• pip - uses certifi cacert.pem by default - this is particularly painful in orgs where a custom package registry is using a cert which organizations tend to sign with their org root CA and is trusted by employee computers.
• pre-commit - uses certifi cacert.pem by default
• sshuttle - uses certifi cascert.pem by default
• yamllint - uses certifi cacert.pem by default
• and a lot lot more.

I'm thinking, could these formulas softlink those cacert.pem files to the ca-certificates bundle file instead? That would fix a lot of issues with not using the system trust store, and I believe homebrew as a distro has the responsibility to unify what trust store tools installed with it are using.

I'd be open to experiment with this and fix the packages I'm actively using, maybe even packages I'm not actively using.

[SMillerDev]

I think the first step should be asking certifi if there is a way to get it to respect the system certificates. That would fix all packages in one go.

[Bo98]

Seems reasonable that we could provide a certifi formula that does this with a symlink, yeah. Other Linux distributions either do this or patch certifi.

[reegnz]

Yeah, having a certifi replacement that uses system-store like this: https://github.com/tiran/certifi-system-store

Then there's other formulas like awscli that just do their own thing without certifi... Those are a bigger nut to crack.

[reegnz]

There's also approaches like https://github.com/sethmlarson/truststore, but I doubt it'd work without modifying the application.

[reegnz]

Also it seems like certifi doesn't want to deal with system trust stores at all: certifi/python-certifi#72
So I think Formulae should deal with patching certifi.

Patching doesn't seem to be too difficult, it should be something like:

  def python3
    which("python3.11")
  end

  def post_install
    site_packages = libexec/Language::Python.site_packages(python3)
    (site_packages/"certifi").install_symlink formula["ca-certificates"].pkgetc/"cert.pem" => "cacert.pem"
  end

[alebcay]

Opened Homebrew/homebrew-core#138377 to provide a brewed certifi which can then be used by Python-based formulae.

[reegnz]

Cool, you're awesome @alebcay!

I'm a bit unsure how this would be consumed by the virtualenv of the individual formulae though, but I hope we'll see some example to start migrating formulae.
EDIT: Ah ok, it installs as a system-wide python dependency. Neat.

[reegnz]

I see this got introduced in Homebrew/homebrew-core@caa7423

🎉