As a developer, I can use a custom CA cert to verify SSL connections during builds

[sclevine]

Important for enterprise/corporate users with transparent mitm proxies

When I run pack build --cert /path/to/cert1.pem --cert /path/to/cert2.pem`
And A process in the container reaches out to website using TLS that requires cert1.pem and cert2.pem CA certificates to validate
Then The connection succeeds

Notes

• /etc/ssl/certs
• Need to run update-ca-certificates
• Should we really do this? Poor cross-stack compat

[ekcasey]

@sclevine even if we don't prioritize fixing this in the general case (because of the problems with stack portability), we should probably handle the specific case of providing a custom ca for connections to a registry.

I imagine we could configure that through ggcr and avoid updating the trust store for the entire container.

[josegonzalez]

This is something that would be useful in my work - I work on a product that enterprise customers install and require their own random certs to be injected into the container.

One thing I thought of here was to "pre-prepare" a stack based on the generic stack. If I "detected" that I needed to inject files, I would simply to do so before actually calling pack/lifecycle, and then use the resulting image within the build process. I don't like that since it requires keeping track of an intermediate stack outside of the general tooling, but it could be useful if this isn't built into pack/lifecycle.

Semi-related was #169, which thankfully is implemented :)

[natalieparellano]

Potentially related to #159

[yonkeltron]

Hey this is blocking my company's exploration of this technology due to our internal security policies and their expressions as crypto-cracking countermeasures at the edge. Since this is a non-negotiable thing for us, we are unable to proceed much further without this capability. How can we work to resolve this, knowing fully that we're willing to help test this in a corporate environment where Man-In-The-Middle proxies exist in force?

Anything we can do to help would be awesome.

[zmackie]

An API for this issue is being discussed in buildpacks/rfcs#69

[dfreilich]

There's been recent movement on this (conversation in slack) - either to implement it as an experimental flag in pack, or to allow it via a new rfc for stackpacks (see more here: buildpacks/rfcs#87)

in the short term, can we implement a --cert like thing in Pack, even if it's super hacky. Then if the grand(iose) vision of stackpacks works out, the --cert could use it under the hood and we can rm -rf the hack (and it just becomes a convenience feature of the Pack platform)?

cc @nebhale @jkutner

[jromero]

Following @dfreilich's update, the alignment during working group was to take the following steps:

1. Research what capabilities are available during buildtime within the docker daemon
2. Start early work on stack buildpacks
3. Possible workaround (within pack) is to use R/W volume mounts

[jromero]

Did a bit of research on this recently. I've cataloged a few solutions on a separate repo. The lowest hanging solution with reasonable results would be the support the Read/Write volumes to enable mounting the docker VM certs. It would be analogous to the following command:

docker run \
    --volume="/etc/ssl/certs:/etc/ssl/certs:ro" \
    --volume="/usr/share/ca-certificates/:/usr/share/ca-certificates/:ro" \
    -it --rm \
    gcr.io/paketo-buildpacks/builder:base \
    /bin/bash

[nebhale]

Love it!

[dfreilich]

@jromero Given that #778 was merged in, can we close this? Or should we keep it open to track in case we need other solutions?

[nebhale]

@dfreilich I'd like this left open until I've got a snapshot to test it with.

[noamichael]

Any update on this? I've tried mounting the certificate volume but I'm still getting SSL errors when running behind a corporate proxy. Don't we also need to run update-ca-certificates after mounting the certificate?

[jromero]

@noamichael I can help troubleshoot a little if you want to open a separate issue or reach out via slack.

FWIW, update-ca-certificates shouldn't be necessary but that would depend on the application. The application can be configured to look solely at ca-certificates.crt or the /etc/ssl/certs/ directory. The former is what gets updated via mount and it's by far the most common.

Here's a draft post that has a little more detail. It will likely be moved to buildpacks.io until a more permanent solution is available (such as stack buildpacks).

[jromero]

The Paketo buildpacks team have found a buildpack acceptable solution for this. You can find it here. We recommend other providers to do something similar.

Once stack buildpacks are available additional options for implementation would be available. That said at this point in time this issue has been satisfied.

[fagiani]

@jromero @dfreilich

This combined with the #169 make it perfect for the use case where in order to improve build speed, additionally to the image caching, you can proxy with Squid with SSL Bump or others the most common packages/artifacts for several projects (like apt, npm, yarn, gem, pip, composer, tgz, etc) and locally cache them and avoid re-downloading GBs worth of data and even save some bucks down the road.

I've just tested it and works like a charm.

Keep Rocking!