x509: consul-webhook-cert-manager is unable to put correct caBundle in consul-controller-mutating-webhook-configuration on consul update

[amit106679]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Upgrading any parameter in consul via helm in a kubernetes cluster starts giving error: Internal error occurred: failed calling webhook "mutate-servicerouter.consul.hashicorp.com": Post "https://consul-consul-controller-webhook.consul.svc:443/mutate-v1alpha1-servicerouter?timeout=10s": x509: certificate signed by unknown authority.

Reproduction Steps

Install consul 1.10.2 via helm.
Update any small parameter like affinity in the config yaml and update via helm.

Cause of the issue & Workaround

Updating consul resets caBundle value in consul-controller-mutating-webhook-configuration to the default value. Restarting the consul-webhook-cert-manager deployment mitigates the issue by putting correct value of caBundle again in the above mutatingwebhookconfiguration.

The expected behaviour should be that webhook-cert-manager reloads the caBundle correctly but on upgrade it is not able to till it is restarted.

[lkysow]

I've reproduced this and we're working on a fix now.

[amit106679]

@lkysow thanks for the fix - any timeline when this will be released on helm charts?

Also we faced another issue:
x509: certificate has expired or is not yet valid: current time 2021-11-09T05:37:48Z is after 2021-11-08T19:40:44Z

Please let me know if this is fixed or should I create another ticket for this?

[lkysow]

We should have a release in the next two weeks. For your other issue please open up a new ticket since the certs should still be renewed even with the current release. Please also include which platform you're running on. I've seen something similar when running locally on Docker when I stop and restart the Docker daemon.

…

On Mon, Nov 8, 2021 at 11:39 PM amit106679 ***@***.***> wrote: @lkysow <https://github.com/lkysow> thanks for the fix - any timeline when this will be released on helm charts? Also we faced another issue - x509: certificate has expired or is not yet valid: current time 2021-11-09T05:37:48Z is after 2021-11-08T19:40:44Z - please let me know if this is fixed or should I create another ticket for this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#808 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAH4RPJE7EUNO5ZO5FS5MITULDF4DANCNFSM5G3MEFMA> .