Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Caddy+Smallstep integration (trusted certs for localhost and internal sites) #3021
Thanks to a recent agreement, I'll soon be working on integrating Smallstep into Caddy.
This should allow us to serve HTTPS for localhost and internal sites automatically, much like we already do for public sites via Let's Encrypt. Essentially, we want
In Caddy 1,
In the future, we can probably extend this to properly-managed mTLS that is (near-)fully automatic in a cluster.
This feature is not a release blocker for Caddy 2.0. It may get into 2.1.
This issue is for tracking discussion of the proposal and getting feedback from interested users and companies. Please get involved!
Here are some questions to help bootstrap development discussion:
Hi everyone. I'm Mike, from smallstep.
We're super interested in uncovering use cases for this integration. To help get a conversation going I wanted to drop a couple ideas to gauge interest, and share a few links for anyone who wants to learn more about this stuff.
Integrating Caddy with smallstep's CA (called
Some use cases that we think may be interesting to the Caddy community are:
@mholt I am not totally familiar with all of Caddys capabilities so this may not be totally relevant but in general, I see private PKI as being super important for API Key like use cases (mTLS) and service to service authentication (again mTLS).
As such, the idea of using SmallStep, over ACME, with external account binding, to acquire mTLS certificates makes a lot of sense to me.
You can see Caddy (as an FE) as the API authentication endpoint where mTLS is in use instead of the typical pre-shared key style API authentication, and having clients keeping their mTLS certificates fresh via an exposed smallstep ACME instance.
You can also see the upstream server being authenticated to via mTLS, where a different smallstep instance, again via external account binding or maybe via the traditional ACME challenges.
There is also the case where the upstream is another caddy instance and it is doing the certificate validation, and of course, getting its own certificate via ACME.
In models like this, you can have point-to-point TLS used for all communications via Private or Public TLS.
In the EDU space, we've talked about a binary/service that looks like our provider. A proxy to do DNS challenges to CloudFlare or something so that we can get back valid LE certs. And then on our internal servers, we'd setup certbot to call this proxy and vioala. Magic.
Smallstep looks cool but many SMBs do not have the resources to ship root CAs every where. Remove that friction and we will buy.
I also like @jahands idea; the ability to use Caddy in the DMZ to do ACME challenge-response on behalf of hosts on the inside of a network is interesting (split horizon).
Though I think its a bad practice to use public SSL certs within an enterprise where machines are managed as it exposes the enterprise to associated operational risk and puts a weight on the public PKI (as we have seen with card readers doing the same thing) it's not clear to me UNIs have much of an option. I am sure there are other similar orgs as well as such something like this would be valuable and probably not a big extension to what you already have in Caddy.
If we threat model this, we generally have a flow chart numbered below.
#1 is where most orgs are at #2 is a nice step but open to MiTM #3 Does put the burden on public PKI infra but I think that could be solved too #4 Would be ideal but many can't afford a FTE to support that infra.
If we are trying to make it harder for attackers, we want solutions that scale without a huge FTE and dollar investment. #3 is what I believe is best based on my experience and those who are underfunded who share their perspective at opsecedu.com. I'm open to other ideas that meet the criteria of super simple to install and scale without FTE or large dollar investments.
Going to dump some relevant and some non-relevant info here that @mholt had asked about in a discussion at Opsecedu - sorry if it's a bit scattered. To preface, the discussion was around internal CA's at school districts and possible gaps that smallstep and/or caddy may have that we are currently handling in other ways.
We utilize ADCS internally to issue certificates for things like EAP-TLS, ConfigMgr which requires special purposes and EKU's on the certificates, and other things that are non-HTTPS focused. I don't believe you can request some of these things from Let's Encrypt, and I'm not familiar enough with smallstep to now if this is supported. It would be interesting if an ACME client were able to request these parameters, but that may help move off ADCS.
Another point was that a lot of MDM solutions and other equipment support SCEP but not ACME or maybe the ability to run the ACME client. I hear the frustration from the development side of things is the tooling operations uses is different than theirs, and I don't believe either of these solutions aim to solve that problem. For example, dev may spin up a box and grab a LE cert, but in production, ops may issue an internal cert via ADCS or SCEP where the dev does not have access/control.
To describe the proxy concept @jaredfolkins mentioned, I can explain what we are currently doing. We built what is basically a centralized certbot that grabs LE certificates for most of our servers with a dns-challenge. It then runs install scripts on remote hosts (think scp cert, relaunch daemon, then validate new cert) and has health check components built into the service as well. This maintains all of our LE stuff, but it would be arguably much easier if smallstep were able to act as a certbot server here.
Some mechanism, whether native smallstep or 3rd part, would get LE certificates onto smallstep, and then local ACME clients such as certbot would be able to request those publicly trusted LE certificates from smallstep. This achieves Jared's wishes that everything use the same protocols for requesting certificates, a unified CA for handling all of your certificates, and dev/ops are using the same tooling.
The part I'm unclear of is how we currently use ADCS to issue specific certificates to users and computers based on templates, disallow certain devices from requesting certificates, etc. I'm assuming that would be part of smallstep, but again, I'm just not familiar with it. Ultimately, we have to keep in mind that much of this is more than just HTTPS (we put certs on AD communications, RDP, ConfigMgr agents, SIEM agents, etc), so I'm not sure how much is really applicable to Caddy vs Smallstep vs anything else.
SmallStep is about lowering those costs and complexities so no full time FTE or other cost is required :)
Gonna hop in here and try to summarize & synthesize a little bit, based on what I'm reading above.
Mutual TLS use cases
@rmhrisk it sounds like you're onboard with the two big mTLS use cases, which would be:
For either of these use cases, I think Caddy would have to grow better client certificate / mTLS support. I'm not sure what the current capabilities are in Caddy v2 but, minimally, it'd need to be able to:
"Web PKI" inside the DMZ
I think this is really going to the bread-and-butter / low-hanging-fruit for an initial integration between Caddy & smallstep. Basically, the idea is to bring the standard PKI used by web browsers for HTTPS inside the DMZ. I think the big use cases for this are:
I'm really interested in learning how common / how important these two use cases are, and whether there are additional use cases in this category.
@jaredfolkins your perspective here re: the cost of running an internal PKI is super valuable. Like @rmhrisk said, our mission is to make PKI more accessible. We want to bring that cost down. If you haven't checked out our CA it is pretty easy to get started, and I'd be very interested to hear feedback on how we can make things even easier.
You mentioned that many SMBs don't have the resources to ship root CAs. Is MDM too expensive (I honestly don't know what MDM solutions cost)? If so, you can actually use
Am I understanding the problem correctly? Any thoughts on how we can improve?
Other PKI use cases
@nathanmcnulty I think you're right that these non-HTTPS use cases aren't really relevant for the Caddy+smallstep integration. But we're absolutely interested in addressing these use cases with
Certificate templating to support other purposes & EKUs is on our short-term open source roadmap. So is SCEP support. While it is possible for
I don't think these use cases are really relevant to the Caddy integration, but they are important insofar as they make adopting
We're also working to support a generic templating mechanism so you can totally customize certificate attributes. But I think that'll be for pretty advanced use cases. See smallstep/cli#110 for a discussion. The big blocker on that, at the moment, is figuring out what format to use for the cert template (not sure if there's a standard for that).
Links to commonly used tools, specs / standards, blog posts, etc. that are relevant to any of these use cases would also be super helpful.
@mmalone That all looks great! I do think Microsoft has (unintentionally) made it difficult to fully replace AD CS because of how closely some features are integrated, such as Windows Hello for Business. These are pretty niche to a heavy Windows shop though, so I feel like those folks (like me) would tend to stick with is most documented and are most familiar with anyway.
Having said that, I don't foresee Microsoft adding ACME to AD CS, so there's still a lot to like about smallstep here, especially from a developer standpoint. Microsoft is also doing a better job of supporting 3rd party CA's in Azure/InTune (SCEP currently), so smallstep may be able to serve up those EAP-TLS certificates to InTune devices once you get further down the road :)
Replacing AD/CS is straight forward. I built this previously and that’s exactly what it does: https://www.globalsign.com/en/auto-enrollment-gateway
Microsoft Intune SCEP really is a Microsoft proprietary protocol; they’ve added other stuff to it that’s not standard, the “standard” parts are not necessarily conformant with the SCEP draft either (there is no final RFC) and the CA must ask InTune to validate some proprietary blobs (a mix of XML and JSON as I recall) in the request prior to issuance.
Just as a bit of extra information... I have successfully used certstrap to set up a CA, and generate site certificates for pods in a kubernetes cluster to talk to each other via TLS.
Once you have the CA, you can generate whatever certs you want, for localhost or wherever else, whenever you need them. And since you're generating them yourself, you control the expiration, etc.