Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document HTTPS with the built-in Traefik (LetsEncrypt and existing certs) #117

Open
flxs opened this Issue Mar 2, 2019 · 17 comments

Comments

Projects
None yet
@flxs
Copy link

commented Mar 2, 2019

Is your feature request related to a problem? Please describe.
I can't seem to find a way to get existing certs into the container, or to allow LetsEncrypt certificates to survive pod termination, other than writing my own Traefik deployment to add a PersistentVolume or deploying Consul alongside. It would be neat to have documentation on the "proper" way of doing this (I assume there is one, and I'm just not knowledgeable enough about Kubernetes to find it).

Describe the solution you'd like
Documentation covering HTTPS with the built-in Traefik, preferably with existing certificates and with LetsEncrypt.

Describe alternatives you've considered
I could disable the built-in Traefik and roll my own, or run Consul alongside, but both seem like a lot of effort for something that feels like a base requirement in a great many use cases.

Additional context
None

@ibuildthecloud

This comment has been minimized.

Copy link
Member

commented Mar 3, 2019

@flxs

This comment has been minimized.

Copy link
Author

commented Mar 3, 2019

That looks very helpful, especially with multi-node-node clusters; having certs in secrets definitely makes distributing them easier, but I'm still unsure how to get those k8s-secrets into the built-in Traefik. Would I have to disable the built-in Traefik and deploy my own to mount the certificate secrets into the container? I can't think of another way to do it, am I missing something obvious?

@flxs

This comment has been minimized.

Copy link
Author

commented Mar 4, 2019

Ok, so cert-manager seems to be the way to go, and there seems to be a way of handing certificates into Traefik by means of Ingress attributes, as described in the Traefik docs.

I can't get cert-manager to work on k3s, though. The instructions in the cert-manager docs for installing via helm chart leave me with a cert-manager-webhook pod saying Error: configmaps "extension-apiserver-authentication" not found. Applying an issuer manifest fails due to the webhook not being reachable.

@aaronkjones

This comment has been minimized.

Copy link

commented Mar 7, 2019

I attempted the same and my failure led me here. I will try with "--no-deploy traefik".

@epicfilemcnulty

This comment has been minimized.

Copy link
Contributor

commented Mar 8, 2019

@flxs For what it's worth, you can install cert-manager without webhook: helm install --name cert-manager --namespace cert-manager stable/cert-manager --set webhook.enabled=false.

@aaronkjones

This comment has been minimized.

Copy link

commented Mar 8, 2019

FYI this does not seem to be armhf compatible.
An arm image exists: quay.io/jetstack/cert-manager-controller-arm

@mashedcode

This comment has been minimized.

Copy link

commented Mar 14, 2019

Since cert-manager is an essential component I wonder how I'm supposed to get my Let's Encrypt stuff working on k3s without it.
IIUC cert-manager with webhook does not work since authorization/v1beta1 got removed from k3s.

I've got no clue what the implications of running it without webhook are but the config below does add the webhook too so I must have certainly miss-configured it.

apiVersion: k3s.cattle.io/v1
kind: HelmChart
metadata:
  name: cert-manager
  namespace: kube-system
spec:
  chart: stable/cert-manager
  set:
    webhook.enabled: "false"

Anyone got a working workaround? Blog post appreciated.

@lentzi90

This comment has been minimized.

Copy link

commented Mar 29, 2019

I am successfully running cert-manager from the jetstack helm repository. The chart is located here.

To get it to work on arm64 I'm using these values:

image:
  repository: quay.io/jetstack/cert-manager-controller-arm64
webhook:
  image:
    repository: quay.io/jetstack/cert-manager-webhook-arm64
cainjector:
  image:
    repository: quay.io/jetstack/cert-manager-cainjector-arm64
extraArgs:
  - --acme-http01-solver-image=quay.io/jetstack/cert-manager-acmesolver-arm64:v0.7.0

The extraArgs part is needed because of this issue and should not be needed with the next release.

@giovannicandido

This comment has been minimized.

Copy link

commented Apr 2, 2019

Criating a ClusterIssuer produces error:

Error from server (InternalError): error when creating "issuer.yaml": Internal error occurred: failed calling webhook "clusterissuers.admission.certmanager.k8s.io": an error on the server ("Internal Server Error: "/apis/admission.certmanager.k8s.io/v1beta1/clusterissuers": the server could not find the requested resource") has prevented the request from succeeding

Configs:

kind: HelmChart
metadata:
  name: cert-manager
  namespace: kube-system
spec:
  chart: cert-manager
  version: v0.7.0
  targetNamespace: cert-manager
  repo: https://charts.jetstack.io
  set:
    ingressShim.defaultIssuerName: letsencrypt-prod
    ingressShim.defaultIssuerKind: ClusterIssuer
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
  namespace: cert-manager
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: user@domain.com
    privateKeySecretRef:
      name: letsencrypt-staging
    http01: {}

Changing from ClusterIssuer to normal Issuer works

@giovannicandido

This comment has been minimized.

Copy link

commented Apr 2, 2019

cert-manager webkook logs:

logging error output: "Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1?timeout=32s\": the server could not find the requested resource\n"
 [k3s/v1.13.5 (linux/amd64) kubernetes/256ea73 10.42.0.1:58750]
I0402 20:59:59.820279       1 request.go:942] Request Body: {"kind":"SubjectAccessReview","apiVersion":"authorization.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"spec":{"nonResourceAttributes":{"path":"/apis/admission.certmanager.k8s.io/v1beta1","verb":"get"},"user":"system:anonymous","group":["system:unauthenticated"]},"status":{"allowed":false}}
I0402 20:59:59.820401       1 round_trippers.go:419] curl -k -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJjZXJ0LW1hbmFnZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiY2VydC1tYW5hZ2VyLXdlYmhvb2stdG9rZW4tMnhnOXEiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiY2VydC1tYW5hZ2VyLXdlYmhvb2siLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI2OWMyZjE3MC01NTg5LTExZTktYmU3ZS1lMjc5ZDY0Nzg1MGIiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6Y2VydC1tYW5hZ2VyOmNlcnQtbWFuYWdlci13ZWJob29rIn0.HNaElcVENOmrGRV6qsAVUi9gT-h3vWMPbyZZyVPqPNxpsAWRonOHHEFGgha_fDfXj0ZEWU1wE8gfzY19u3ZwdNAsnr4k_5WzWd5-O6MemIPxxCtvjubw-KFp4_X0Y42U3xDID6Joa1INA3xMyhizIozUBVzaoNj1Hx0dl8uyGM7FtjtDgM1cT2RQVDp8LFsyVVWMTTRxGXL-E8JWzzGulEPMCsYlNwOoJ3Dq7NpDR0ONB4tEbde6k0EbMsXvXbUV1Kj9zBSHU3pN-KiYrauACI5yAWwBWO6O9WOK1wpRSFbiyj8L4Ez0Dan5b9P8x1Q50VyZzt8hpoBs8JDZZmKOEw" -H "User-Agent: image.app_linux-amd64.binary/v0.0.0 (linux/amd64) kubernetes/$Format" 'https://10.43.0.1:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews'
I0402 20:59:59.821630       1 round_trippers.go:438] POST https://10.43.0.1:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews 404 Not Found in 1 milliseconds
I0402 20:59:59.821644       1 round_trippers.go:444] Response Headers:
I0402 20:59:59.821648       1 round_trippers.go:447]     Content-Length: 174
I0402 20:59:59.821652       1 round_trippers.go:447]     Content-Type: application/json
I0402 20:59:59.821656       1 round_trippers.go:447]     Date: Tue, 02 Apr 2019 20:59:59 GMT
I0402 20:59:59.821696       1 request.go:942] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the server could not find the requested resource","reason":"NotFound","details":{},"code":404}
E0402 20:59:59.821789       1 webhook.go:192] Failed to make webhook authorizer request: the server could not find the requested resource
E0402 20:59:59.821941       1 errors.go:77] the server could not find the requested resource
I0402 20:59:59.821969       1 wrap.go:47] GET /apis/admission.certmanager.k8s.io/v1beta1?timeout=32s: (1.915603ms) 500
goroutine 428 [running]:
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc0003962a0, 0x1f4)
        vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:204 +0xd2
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc0003962a0, 0x1f4)
        vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:183 +0x35
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc00023fe00, 0x1f4)
        vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:205 +0xaf
net/http.Error(0x7f2b1ceea2c0, 0xc00000c320, 0xc0000886c0, 0x81, 0x1f4)
        GOROOT/src/net/http/server.go:1976 +0xda
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.InternalError(0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00, 0x7f2b1ceea440, 0xc0002e80e0)
        vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters/errors.go:75 +0x126
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00)
        vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:69 +0x1ed
net/http.HandlerFunc.ServeHTTP(0xc0003dea80, 0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00)
        GOROOT/src/net/http/server.go:1964 +0x44
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00)
        vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:160 +0x434
net/http.HandlerFunc.ServeHTTP(0xc000609410, 0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00)
        GOROOT/src/net/http/server.go:1964 +0x44
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00)
        vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:50 +0x1eeb
net/http.HandlerFunc.ServeHTTP(0xc0003deac0, 0x7f2b1ceea2c0, 0xc00000c320, 0xc000467c00)
        GOROOT/src/net/http/server.go:1964 +0x44
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f2b1ceea2c0, 0xc00000c320, 0xc000467b00)
        vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:81 +0x476
net/http.HandlerFunc.ServeHTTP(0xc0004c70e0, 0x7f2b1ceea2c0, 0xc00000c320, 0xc000467b00)
        GOROOT/src/net/http/server.go:1964 +0x44
github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc0004a2360, 0xc0004c1ce0, 0x196a2a0, 0xc00000c320, 0xc000467b00)
        vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:108 +0xb3
created by github.com/jetstack/cert-manager/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
        vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:97 +0x1b0

logging error output: "Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1?timeout=32s\": the server could not find the requested resource\n"
 [k3s/v1.13.5 (linux/amd64) kubernetes/256ea73/controller-discovery 10.42.0.1:58750]
@giovannicandido

This comment has been minimized.

Copy link

commented Apr 4, 2019

Update: Issuer works because I removed the webhook validation from the namespace. ClusterIssuer keeps validating.
I did try to disable in helm installation and it does not disable the thing :-). Will check that later.

@m4rcu5

This comment has been minimized.

Copy link

commented Apr 6, 2019

I did try to disable in helm installation and it does not disable the thing :-). Will check that later.

I'd like to chime in here.

I also try to install the cert-manager without web-hook and that should be possible by passing --set webhook.enabled=false to the helm installer.

So I took @giovannicandido snipped as base (which he posted right around the same time I figured out what the Helmchart variables were from the source code. some docs around that feature would be nice 😄 )

apiVersion: k3s.cattle.io/v1
kind: HelmChart
metadata:
  name: cert-manager
  namespace: kube-system
spec:
  chart: cert-manager
  version: v0.7.0
  targetNamespace: cert-manager
  repo: https://charts.jetstack.io

And to disable the webhook, I added:

set:
  webhook.enabled: false

This results in nothing at all. The Helmchart does not even seem to be processed, but no error is thrown as well.

Next up I tried:

set:
  webhook.enabled: "false"

This was interperated by k3s as a string and resulted in the following log lines in the helm-install pod:

+ helm install --name cert-manager cert-manager --namespace cert-manager --repo https://charts.jetstack.io --version v0.7.0 --set-string webhook.enabled=false
2019/04/06 14:51:57 Warning: Condition path 'webhook.enabled' for chart webhook returned non-bool value

I have also tried with 0 as argument, but the same warning applies.

Is there any way to pass a boolean value to the Helm installer process using the k3s.cattle.io/v1 API?

@thedirtymexican

This comment has been minimized.

Copy link

commented Apr 16, 2019

I am also trying to get cert-manager to run on k3s, after following the instructions on https://docs.cert-manager.io/en/latest/getting-started/install.html#installing-with-helm I was able to:

  1. Apply CRDs:
    kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/cert-manager.yaml
  2. install cert-manager via helm (disabling webhooks):
    helm install --name cert-manager --namespace cert-manager --version v0.7.0 jetstack/cert-manager --set webhook.enabled=false
  3. Apply Issuer (ClusterIssuer always fails)

After updating my Ingress to use the newly configured Issuer I can see the following error in the logs:
ingress-shim controller: Re-queuing item "<namespace>/<ingress-name>" due to error processing: Internal error occurred: failed calling webhook "certificates.admission.certmanager.k8s.io": an error on the server ("Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1/certificates\": the server could not find the requested resource") has prevented the request from succeeding

@jaigouk

This comment has been minimized.

Copy link

commented Apr 17, 2019

Hello,

I have been trying to install cert-manager and monitoring this thread and this one. I also referenced this issue in jetstack/cert-manager repo.

I have 4 tinker boards and I updated k3s to v0.4.0 yesterday.

I put following yaml as /var/lib/rancher/k3s/server/manifests/cert-manager.yml. I used sudo su to access that dir. After that, I rebooted my tinkerboard to make it sure that k3s picks up the file.

apiVersion: k3s.cattle.io/v1
kind: HelmChart
metadata:
  name: cert-manager
  namespace: kube-system
spec:
  chart: stable/cert-manager
  valuesContent: |-
    image:
      repository: quay.io/jetstack/cert-manager-controller-arm
      tag: v0.7.0
      pullPolicy: IfNotPresent
    webhook:
      enabled: false

Here is the result from helm-install-cert-manager job in kube-system namespace

NAMESPACE: kube-system
STATUS: DEPLOYED
RESOURCES:
==> v1/ServiceAccount
NAME          SECRETS  AGE
cert-manager  1        1s
==> v1beta1/ClusterRole
NAME          AGE
cert-manager  1s
==> v1/ClusterRole
NAME               AGE
cert-manager-view  1s
cert-manager-edit  1s
==> v1beta1/ClusterRoleBinding
NAME          AGE
cert-manager  1s
==> v1beta1/Deployment
NAME          DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
cert-manager  1        0        0           0          0s
==> v1/Pod(related)
NAME                           READY  STATUS   RESTARTS  AGE
cert-manager-666775646b-wm28f  0/1    Pending  0         0s
NOTES:
cert-manager has been deployed successfully!
In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).
More information on the different types of issuers and how to configure them
can be found in our documentation:
https://cert-manager.readthedocs.io/en/latest/reference/issuers.html
For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:
https://cert-manager.readthedocs.io/en/latest/reference/ingress-shim.html
**This Helm chart is deprecated**.
All future changes to the cert-manager Helm chart should be made in the
official repository: https://github.com/jetstack/cert-manager/tree/master/deploy.
The latest version of the chart can be found on the Helm Hub: https://hub.helm.sh/charts/jetstack/cert-manager.
+ exit

And the deployment status says that it is healthy.

Screenshot 2019-04-17 at 23 12 47

@lentzi90

This comment has been minimized.

Copy link

commented Apr 22, 2019

I cannot reproduce the webhook issue. Is it only a problem if you install the chart the "k3s way"? (By putting it in /var/lib/rancher/k3s/server/manifests/cert-manager.yml?)

This is how I install it with webhooks.

# Install helm first by downloading the binary from their release page: https://github.com/helm/helm/releases

# Create service account and RBAC resources for tiller
kubectl apply -f - <<EOF                                                                                                                                                                                                                                   
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system                                                                                            
EOF

# Initialize tiller
KUBECONFIG=/etc/rancher/k3s/k3s.yaml
helm init --service-account tiller

# The following commands are directly from the cert-manager installation guide for helm
# https://docs.cert-manager.io/en/latest/getting-started/install.html#steps

# Install the CustomResourceDefinition resources separately
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/00-crds.yaml

# Create the namespace for cert-manager
kubectl create namespace cert-manager

# Label the cert-manager namespace to disable resource validation
kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io

# Update your local Helm chart repository cache
helm repo update

# Install the cert-manager Helm chart
helm install \
  --name cert-manager \
  --namespace cert-manager \
  --version v0.7.0 \
  jetstack/cert-manager

# Wait for the pods to be ready here...
kubectl get pods --namespace cert-manager

# Create certificates, Issuer and ClusterIssuer to test deployment
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager-test
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
  name: test-selfsigned
  namespace: cert-manager-test
spec:
  selfSigned: {}
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: selfsigned-cert
  namespace: cert-manager-test
spec:
  commonName: example.com
  secretName: selfsigned-cert-tls
  issuerRef:
    name: test-selfsigned
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: test-selfsigned-cluster
spec:
  selfSigned: {}
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: selfsigned-cert-cluster
  namespace: cert-manager-test
spec:
  commonName: example.com
  secretName: selfsigned-cert-tls-cluster
  issuerRef:
    name: test-selfsigned-cluster
    kind: ClusterIssuer
EOF

# Check that certs are issued
kubectl describe certificate -n cert-manager-test
@thedirtymexican

This comment has been minimized.

Copy link

commented Apr 24, 2019

Hi @lentzi90,

I followed your same steps. Cert-manager installs correctly, but never issues a certificate.

kubectl apply -f - <<EOF
> apiVersion: v1
> kind: Namespace
> metadata:
>   name: cert-manager-test
> ---
> apiVersion: certmanager.k8s.io/v1alpha1
> kind: Issuer
> metadata:
>   name: test-selfsigned
>   namespace: cert-manager-test
> spec:
>   selfSigned: {}
> ---
> apiVersion: certmanager.k8s.io/v1alpha1
> kind: Certificate
> metadata:
>   name: selfsigned-cert
>   namespace: cert-manager-test
> spec:
>   commonName: example.com
>   secretName: selfsigned-cert-tls
>   issuerRef:
>     name: test-selfsigned
> ---
> apiVersion: certmanager.k8s.io/v1alpha1
> kind: ClusterIssuer
> metadata:
>   name: test-selfsigned-cluster
> spec:
>   selfSigned: {}
> ---
> apiVersion: certmanager.k8s.io/v1alpha1
> kind: Certificate
> metadata:
>   name: selfsigned-cert-cluster
>   namespace: cert-manager-test
> spec:
>   commonName: example.com
>   secretName: selfsigned-cert-tls-cluster
>   issuerRef:
>     name: test-selfsigned-cluster
>     kind: ClusterIssuer
> EOF
namespace/cert-manager-test created
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "issuers.admission.certmanager.k8s.io": an error on the server ("Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1/issuers\": the server could not find the requested resource") has prevented the request from succeeding
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "certificates.admission.certmanager.k8s.io": an error on the server ("Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1/certificates\": the server could not find the requested resource") has prevented the request from succeeding
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "clusterissuers.admission.certmanager.k8s.io": an error on the server ("Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1/clusterissuers\": the server could not find the requested resource") has prevented the request from succeeding
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "certificates.admission.certmanager.k8s.io": an error on the server ("Internal Server Error: \"/apis/admission.certmanager.k8s.io/v1beta1/certificates\": the server could not find the requested resource") has prevented the request from succeeding
@lentzi90

This comment has been minimized.

Copy link

commented Apr 24, 2019

I was able to reproduce the same error you got @thedirtymexican but only when running all the commands at once with a script. To fix it, I simply rerun the last part (after create namespace). Then it worked.

I was also able to reproduce this problem in minikube, so it doesn't seem to be related to k3s.

This seem to be a timing issue to me. If the Issuer is created too fast it fails. It is not enough to wait for the pods to become ready either, I tried this and still got the error.
It is curious though that the issuers that the helm chart installs always works...

I also managed to get another error while testing, where it succeeds with the first cert and fails with the second:

namespace/cert-manager-test created
issuer.certmanager.k8s.io/test-selfsigned created
certificate.certmanager.k8s.io/selfsigned-cert created
clusterissuer.certmanager.k8s.io/test-selfsigned-cluster created
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "certificates.admission.certmanager.k8s.io": 0-length response with status code: 200 and content type: text/plain; charset=utf-8

To help with debugging I created a Vagrantfile and a script for quickly installing cert-manager, issuer and certificate.
See below.
(Github wouldn't allow me to upload without changing to .txt, please remove the suffix before using.)
Vagrantfile.txt
cert-manager-test.sh.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.