Subject of the issue
I fully admit to not being a k8s expert so this could very well be operator error. I have a fork with a fix that is working in my environment but I wanted to file this issue first in case I am missing something obvious. I am trying to deploy autocert in a k3s cluster and ran into problems that were difficult to diagnose.
The autocert controller uses k8s.io/api/admission/v1beta1 types for AdmissionReview and AdmissionResponse in controller/main.go. The response is marshaled as:
v1beta1.AdmissionReview{Response: response}
without setting TypeMeta (apiVersion and kind). This produces a JSON response like:
{"response":{"uid":"...","allowed":true,"patch":"...","patchType":"JSONPatch"}}
The Kubernetes API server (v1.22+, where v1beta1 admission is removed) expects a complete AdmissionReview envelope:
{"apiVersion":"admission.k8s.io/v1","kind":"AdmissionReview","response":{...}}
Additionally, the MutatingWebhookConfiguration (both in install/02-autocert.yaml and the Helm chart) declares admissionReviewVersions: ["v1beta1"], but v1beta1 was removed from the admission API in Kubernetes 1.22.
Environment
- Kubernetes: v1.34.6+k3s1
- autocert: v0.20.6
- step-ca: v0.30.1
Steps to reproduce
- Deploy step-ca and autocert controller on Kubernetes 1.22+
- Label a namespace:
kubectl label namespace default autocert.step.sm=enabled
- Create a pod with annotation:
autocert.step.sm/name: test.default.svc.cluster.local
- Observe: pod is created with only the original container (no bootstrapper init container, no renewer sidecar)
- Check autocert controller logs: "Generated patch" is logged, confirming the webhook processed the request
Actual behaviour
Direct curl to the webhook endpoint confirms the response lacks the AdmissionReview wrapper:
kubectl run curl-test --image=curlimages/curl --restart=Never -n step --rm -i -- \
curl -sk https://autocert.step.svc.cluster.local/mutate \
-H 'Content-Type: application/json' \
-d '{"apiVersion":"admission.k8s.io/v1","kind":"AdmissionReview","request":{"uid":"test","kind":{"group":"","version":"v1","kind":"Pod"},"resource":{"group":"","version":"v1","resource":"pods"},"namespace":"default","operation":"CREATE","object":{"apiVersion":"v1","kind":"Pod","metadata":{"generateName":"test-","namespace":"default","annotations":{"autocert.step.sm/name":"test.default.svc.cluster.local"}},"spec":{"containers":[{"name":"test","image":"busybox"}]}}}}'
Response (missing apiVersion and kind):
{"response":{"uid":"test","allowed":true,"patch":"...","patchType":"JSONPatch"}}
Additional context
I have a fix in my fork that I've tested and is working:
master...barrettc:autocert:fix/admission-v1-k8s-1.22-compat
I can submit a PR if this is the right approach. My spidey sense is telling me something more obvious is going on because k8s 1.22 was released in 2021 and I would have expected others to have this issue before now.
Subject of the issue
I fully admit to not being a k8s expert so this could very well be operator error. I have a fork with a fix that is working in my environment but I wanted to file this issue first in case I am missing something obvious. I am trying to deploy autocert in a k3s cluster and ran into problems that were difficult to diagnose.
The autocert controller uses
k8s.io/api/admission/v1beta1types forAdmissionReviewandAdmissionResponseincontroller/main.go. The response is marshaled as:without setting
TypeMeta(apiVersionandkind). This produces a JSON response like:{"response":{"uid":"...","allowed":true,"patch":"...","patchType":"JSONPatch"}}The Kubernetes API server (v1.22+, where v1beta1 admission is removed) expects a complete
AdmissionReviewenvelope:{"apiVersion":"admission.k8s.io/v1","kind":"AdmissionReview","response":{...}}Additionally, the MutatingWebhookConfiguration (both in
install/02-autocert.yamland the Helm chart) declaresadmissionReviewVersions: ["v1beta1"], but v1beta1 was removed from the admission API in Kubernetes 1.22.Environment
Steps to reproduce
kubectl label namespace default autocert.step.sm=enabledautocert.step.sm/name: test.default.svc.cluster.localActual behaviour
Direct curl to the webhook endpoint confirms the response lacks the AdmissionReview wrapper:
kubectl run curl-test --image=curlimages/curl --restart=Never -n step --rm -i -- \ curl -sk https://autocert.step.svc.cluster.local/mutate \ -H 'Content-Type: application/json' \ -d '{"apiVersion":"admission.k8s.io/v1","kind":"AdmissionReview","request":{"uid":"test","kind":{"group":"","version":"v1","kind":"Pod"},"resource":{"group":"","version":"v1","resource":"pods"},"namespace":"default","operation":"CREATE","object":{"apiVersion":"v1","kind":"Pod","metadata":{"generateName":"test-","namespace":"default","annotations":{"autocert.step.sm/name":"test.default.svc.cluster.local"}},"spec":{"containers":[{"name":"test","image":"busybox"}]}}}}'Response (missing
apiVersionandkind):{"response":{"uid":"test","allowed":true,"patch":"...","patchType":"JSONPatch"}}Additional context
I have a fix in my fork that I've tested and is working:
master...barrettc:autocert:fix/admission-v1-k8s-1.22-compat
I can submit a PR if this is the right approach. My spidey sense is telling me something more obvious is going on because k8s 1.22 was released in 2021 and I would have expected others to have this issue before now.