SSL certificate reload without restart

[Afterglow]

I posted on the google groups and was directed here so here goes.

We use anchor by the openstack project to issue some of our remote nodes not within our own network with short lived SSL certificates. The result is that we have to restart consul frequently to pick up the new certificates. Is there any means to reload the SSL certificates without restarting consul completely? If not would this be considered as a feature request?

https://groups.google.com/forum/#!msg/consul-tool/SdHt8UJ8b_g/RpVnnGt5AAAJ

thanks!
Paul

[tniswong]

Would very much like to see this implemented ala reloadable configuration. I.e. Send a SIGHUP to reload TLS Cert/Key/CA's.

This enables automated cert rotation sans downtime, an effective compliment to the Autopilot Pattern via ContainerPilot.

[freman]

This explains why our vault + consul hasn't been behaving.

[shayangz]

For what is worth, I would vote for the timing/priority of this work being pushed forward. Full restart is very disruptive.

[freman]

Yeh that'd be nice, my confusion came from the fact that I'd written a nice tool to manage the certificates and had figured out the vault reload process and copied that to consul figuring "this is all hashicorp, they'd do it the same way" then 30 days later all my stuff stops working :(

Ok googles oh, it's consul reload... tweak all the things... 30 days later, same issue.

More googles oh, it's not currently possible without restart... this conversation :(

[tniswong]

Yeah. We worked around it by scripting a coordinated cluster failure/outage recovery...

https://www.consul.io/docs/guides/outage.html

[shayangz]

@slackpad: now that 0.9 is out the door do you think this can get on the roadmap?

[SoMuchToGrok]

This feature is especially important when using nomad with TLS.

Currently, the built-in consul agent -> nomad health check is over HTTPS. When a new root CA is generated, and nomad is issued a server certificate signed by that new root CA, the consul agent requires a refresh of its trust anchors (which is currently a full restart of the service). Without the consul trust anchors being refreshed, the nomad health checks will fail. I would love to not restart consul, but just send it a SIGHUP instead.

Nomad is currently implementing this:
hashicorp/nomad#3479

[slackpad]

We shall draft off of Nomad's work for this one :-)

[tniswong]

No love for this one? :(

[freman]

Literally causing us to abandon vault and ssl

[l33t0]

Any updates?

[shayangz]

Just re-iterating how valuable this feature would be. 👍

[freman]

Pretty sure I could have forked and patched it by now... just sayin...

[banks]

@freman PRs are welcome! :)

We're aware this is high on people's wishlist, as always we have a juggling act with various prorities. Thanks for your patience.

[SoMuchToGrok]

To add some more input to this ticket, I think it would be incredibly useful if this was extended to include refreshing the root CA as well, not just the server certificate. One of the benefits of using Vault is how trivial it makes short-lived PKI certs, all the way up to the root CA.

[patelpu94]

Any update on this?

[tristanmorgan]

The PR by @akshayganeshen is working well in a test environment for me but I need an Enterprise (pro) release of this for my client.

[banks]

Thanks again for the patience folks. We are still working towards this as a TODO item.

I hope my comment on the PR is helpful context.

[kikitux]

@tristanmorgan enterprise feedback have different channels, please feel free to reach me at alvaro hashicorp.com

[lmb]

We're currently evaluating Consul, and this issue has popped up. We use short-ish lived certificates and can't afford to refresh these manually.

What are the chances of Step 1 in the comment from @banks happening without the fancier stuff?

[banks]

@lmb the plans around TLS are being worked on now. Rebuilding this one way while in the misdst of designing a better overall solution doesn't seem like the wisest use of the resources we have but the next major release cycle has the whole TLS setup firmly in it's sights and will result in at least basic reloading working in a way that is compatible with our future plans -- hopefully more!

[hanshasselberg]

We merged support for reloading TLS config into master: #5419.