Feature: Add support for per certificate ssl configuration

[techmunk]

HA Proxy supports optional ssl configuration and sni filters per certificate so you can for example have client required verification on certain SSL hosts only. The documentation is at https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#5.1-crt-list

A sample crt-list.txt file I have created to test this looks like:

/certs/nginx-test-2.example.com.pem [ca-file /certs/tmp/ca.crt verify required]
/certs/nginx-test-1.example.com.pem

As expected, test-1 can be accessed freely by anyone; test-2 requires valid client certificates to be present for the service to be hit.

At the moment I see the SSL certificate list is created by adding any files from /certs. Given the above configuration can get quite complex, I propose the addition of the option DF_CRT_LIST_PATH to DFP. When set, DFP will no longer try and create the crt-list.txt file and will instead rely on outside sources to correctly create that file.

This is just one possible solution. I'd love the ability to achieve per certificate sslbind options and if anyone has a better or more convenient solution I'd be happy to go with that.

I'm aware there are a few other places that SSL certificates are read/used in DFP but I confess I am not fully aware of the feature set those instances provide. The above proposal does not really consider how to handle certificate addition or removal from the crt-list. I'd assume that whatever system creates the crt-list would also need to trigger a reload ?

What are peoples thoughts ? Does this seem like a feature we could viably add to DFP?

[techmunk]

Digging a bit more, perhaps this is a feature that can be combined with serviceCert on a per service level. I'm not quite sure how serviceCert works, but it seems promising at a glance. Would also need the ability to set a CA per service as well. This might be less practical... not sure.

[thomasjpfan]

Looks like you want to add configurations such as [ca-file /certs/tmp/ca.crt verify required] to the end of some domains. One way to implement this feature is to add a com.df.crtListConfig=[ca-file /certs/tmp/ca.crt verify required] to the services you want to additional configuration.

If you were to use ca-file /certs/tmp/ca.crt, how do you plan to include /certs/tmp/ca.crt in DFP?

[techmunk]

Yes, that's pretty much exactly what we want to do. It would also be nice to be able to use the sni filter as well i.e. [ca-file /certs/tmp/ca.crt verify required] *.domain.tld !secure.domain.tld. It may also be desirable to be able to have multiple of these per service, and would probably need a way to relate that to specific certs within the /certs directory.

With regards to how we plan to include the ca certs, they were being manually (i.e. outside of DFP) mounted in the DFP instance. This way we can use the same CA across multiple different services if required.
I suppose you could add the CA per service, but it's not our current intended workflow. Given the cost of setting up client certificates I don't think it's a bit ask to have added to DFP outside of the DFP configuration, though I'm not sure how haproxy handles "missing" files. I'd assume it would be somewhat graceful.

[thomasjpfan]

After some thinking, I do not think a service level, com.df.crtListConfig, is a good option for this feature.

When using Put Certificate to insert the certificate, I can add a parameter crtListConfig to the PUT request. This way DFP can add the additional configuration option to crt-list.txt.

When using Secrets to configure certificates, I am still thinking of a way to pass the crtListConfig information.

I am trying to avoid adding DF_CRT_LIST_PATH, because a user would need to know where certs are stored in DFP.

[techmunk]

I agree that the crtListConfig needs to more closely match the certificates rather than the service.
My initial thoughts on this were to add metadata files next to each certificate e.g. cert-1.pem, cert-1.meta or even meta-cert-1.pem. This could be difficult to achieve backwards compatibility though, as instead of just loading every file in the certs directory you'd have to interrogate the actual names to see what to do with it. There's also concern around adding the contents of a .meta file to each certificate line. Accidental new lines at the end of the .meta file could break the output if not handled.

I am trying to avoid adding DF_CRT_LIST_PATH, because a user would need to know where certs are stored in DFP.

Is this really an issue though ? I assumed the feature would be more of an advanced option for people that need the ability to adjust per certificate options, which means they've most likely bind mounted in the /certs directory anyway as they either want to change something like the curves option, or want to add client certificates into the mix but either not as a global option on the port 443 bind, or they have multiple CA's which they need to support.

I would imagine for this option something like a DFP-LE image would be responsible for creating the crt list file.

Our usage for example we have an image not unlike n1b0r/docker-flow-proxy-letsencrypt. Instead of using LE we have an API implementation to a paid SSL provider. The net outcome is the same though.
A service is configured with something like --label com.df.letsencrypt.host=whoami.example.com and a certificate appears in a mounted volume which is writable by the SSL provider image but mounted at /certs as readonly to DFP.
We're currently bind mounting the /cfg/crt-list.txt file readonly within DFP, which means the file needs to be on every swarm node that DFP could run on which means it's actually managed outside of the swarm as it is currently in an NFS mount. If we could NFS mount to a single file we'd just be doing that. Alas, this does not work, and we can't NFS mount /cfg readonly as DFP needs to write to that path. Which is how we arrived at the proposed solution above; change the path to the crt-list file to a place where we can use an NFS mount within the service file rather than needing to remember to do it for each node added to the swarm that may or may not be running DFP at any point in time.

For our purposes just being able to change the crt-list path and have DFP assume that all things SSL are already handled and not to do anything except print out the bind options would be ideal.
I understand however that you need to consider how other people might use this feature and want to head off any unexpected issue or support requests that users past or present may encounter by such a change.

If this is a feature you'd rather not add we totally understand. It's quite an advanced feature and we're more than capable of building a patched version of DFP for our purposes. We just wanted to try and avoid doing that.

As it stands we cannot immediately think of a way to make this a simple to use turn key feature as the way DFP currently works seems adequate for 99% of use cases. It's only when you need granular control per certificate that it really becomes an issue.

[thomasjpfan]

Implementing CRT_LIST_PATH would be a good solution for your use case. I propose the following:

1. By default, CRT_LIST_PATH will be undefined.
2. When CRT_LIST_PATH is defined, DFP will not generate the /cfg/crt-list.txt file.
3. DFP will use CRT_LIST_PATH in its haproxy configuration.

What do you think?

[techmunk]

What you have described above would work well for us yes.

[thomasjpfan]

This feature was added to dockerflow/docker-flow-proxy:18.07.18-74. Please test it out and see if it works for your use case.

[techmunk]

Thanks @thomasjpfan . Tested and works perfectly.