Document HTTPS with the built-in Traefik (LetsEncrypt and existing certs) #117
Comments
|
Have you tried https://github.com/jetstack/cert-manager? |
|
That looks very helpful, especially with multi-node-node clusters; having certs in secrets definitely makes distributing them easier, but I'm still unsure how to get those k8s-secrets into the built-in Traefik. Would I have to disable the built-in Traefik and deploy my own to mount the certificate secrets into the container? I can't think of another way to do it, am I missing something obvious? |
|
Ok, so cert-manager seems to be the way to go, and there seems to be a way of handing certificates into Traefik by means of Ingress attributes, as described in the Traefik docs. I can't get cert-manager to work on k3s, though. The instructions in the cert-manager docs for installing via helm chart leave me with a |
|
I attempted the same and my failure led me here. I will try with "--no-deploy traefik". |
|
@flxs For what it's worth, you can install |
|
|
|
Since cert-manager is an essential component I wonder how I'm supposed to get my Let's Encrypt stuff working on k3s without it. I've got no clue what the implications of running it without webhook are but the config below does add the webhook too so I must have certainly miss-configured it. Anyone got a working workaround? Blog post appreciated. |
erikwilson
added
kind/documentation
|
I am successfully running cert-manager from the jetstack helm repository. The chart is located here. To get it to work on arm64 I'm using these values: The |
|
Criating a ClusterIssuer produces error:
Configs: kind: HelmChart
metadata:
name: cert-manager
namespace: kube-system
spec:
chart: cert-manager
version: v0.7.0
targetNamespace: cert-manager
repo: https://charts.jetstack.io
set:
ingressShim.defaultIssuerName: letsencrypt-prod
ingressShim.defaultIssuerKind: ClusterIssuer
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
namespace: cert-manager
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: user@domain.com
privateKeySecretRef:
name: letsencrypt-staging
http01: {}Changing from ClusterIssuer to normal Issuer works |
|
cert-manager webkook logs: |
|
Update: Issuer works because I removed the webhook validation from the namespace. ClusterIssuer keeps validating. |
I'd like to chime in here. I also try to install the So I took @giovannicandido snipped as base (which he posted right around the same time I figured out what the Helmchart variables were from the source code. some docs around that feature would be nice 😄 ) And to disable the webhook, I added: This results in nothing at all. The Helmchart does not even seem to be processed, but no error is thrown as well. Next up I tried: This was interperated by I have also tried with Is there any way to pass a boolean value to the Helm installer process using the |
|
I am also trying to get cert-manager to run on k3s, after following the instructions on https://docs.cert-manager.io/en/latest/getting-started/install.html#installing-with-helm I was able to:
After updating my Ingress to use the newly configured Issuer I can see the following error in the logs: |
|
Hello, I have been trying to install cert-manager and monitoring this thread and this one. I also referenced this issue in jetstack/cert-manager repo. I have 4 tinker boards and I updated k3s to v0.4.0 yesterday. I put following yaml as apiVersion: k3s.cattle.io/v1
kind: HelmChart
metadata:
name: cert-manager
namespace: kube-system
spec:
chart: stable/cert-manager
valuesContent: |-
image:
repository: quay.io/jetstack/cert-manager-controller-arm
tag: v0.7.0
pullPolicy: IfNotPresent
webhook:
enabled: falseHere is the result from helm-install-cert-manager job in kube-system namespace NAMESPACE: kube-system
STATUS: DEPLOYED
RESOURCES:
==> v1/ServiceAccount
NAME SECRETS AGE
cert-manager 1 1s
==> v1beta1/ClusterRole
NAME AGE
cert-manager 1s
==> v1/ClusterRole
NAME AGE
cert-manager-view 1s
cert-manager-edit 1s
==> v1beta1/ClusterRoleBinding
NAME AGE
cert-manager 1s
==> v1beta1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
cert-manager 1 0 0 0 0s
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
cert-manager-666775646b-wm28f 0/1 Pending 0 0s
NOTES:
cert-manager has been deployed successfully!
In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).
More information on the different types of issuers and how to configure them
can be found in our documentation:
https://cert-manager.readthedocs.io/en/latest/reference/issuers.html
For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:
https://cert-manager.readthedocs.io/en/latest/reference/ingress-shim.html
**This Helm chart is deprecated**.
All future changes to the cert-manager Helm chart should be made in the
official repository: https://github.com/jetstack/cert-manager/tree/master/deploy.
The latest version of the chart can be found on the Helm Hub: https://hub.helm.sh/charts/jetstack/cert-manager.
+ exitAnd the deployment status says that it is healthy.
|
|
I cannot reproduce the webhook issue. Is it only a problem if you install the chart the "k3s way"? (By putting it in This is how I install it with webhooks. # Install helm first by downloading the binary from their release page: https://github.com/helm/helm/releases
# Create service account and RBAC resources for tiller
kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
EOF
# Initialize tiller
KUBECONFIG=/etc/rancher/k3s/k3s.yaml
helm init --service-account tiller
# The following commands are directly from the cert-manager installation guide for helm
# https://docs.cert-manager.io/en/latest/getting-started/install.html#steps
# Install the CustomResourceDefinition resources separately
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/00-crds.yaml
# Create the namespace for cert-manager
kubectl create namespace cert-manager
# Label the cert-manager namespace to disable resource validation
kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true
# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io
# Update your local Helm chart repository cache
helm repo update
# Install the cert-manager Helm chart
helm install \
--name cert-manager \
--namespace cert-manager \
--version v0.7.0 \
jetstack/cert-manager
# Wait for the pods to be ready here...
kubectl get pods --namespace cert-manager
# Create certificates, Issuer and ClusterIssuer to test deployment
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager-test
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
name: test-selfsigned
namespace: cert-manager-test
spec:
selfSigned: {}
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: selfsigned-cert
namespace: cert-manager-test
spec:
commonName: example.com
secretName: selfsigned-cert-tls
issuerRef:
name: test-selfsigned
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
name: test-selfsigned-cluster
spec:
selfSigned: {}
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: selfsigned-cert-cluster
namespace: cert-manager-test
spec:
commonName: example.com
secretName: selfsigned-cert-tls-cluster
issuerRef:
name: test-selfsigned-cluster
kind: ClusterIssuer
EOF
# Check that certs are issued
kubectl describe certificate -n cert-manager-test |
|
Hi @lentzi90, I followed your same steps. Cert-manager installs correctly, but never issues a certificate. |
|
I was able to reproduce the same error you got @thedirtymexican but only when running all the commands at once with a script. To fix it, I simply rerun the last part (after create namespace). Then it worked. I was also able to reproduce this problem in minikube, so it doesn't seem to be related to k3s. This seem to be a timing issue to me. If the Issuer is created too fast it fails. It is not enough to wait for the pods to become ready either, I tried this and still got the error. I also managed to get another error while testing, where it succeeds with the first cert and fails with the second: To help with debugging I created a Vagrantfile and a script for quickly installing cert-manager, issuer and certificate. |
|
I've not managed to repro this myself on a brand new k3s cluster; it just seems to work out of the box if you follow their instructions, most importantly step 1! My steps were:
apiVersion: k3s.cattle.io/v1
kind: HelmChart
metadata:
namespace: kube-system
name: cert-manager
spec:
chart: cert-manager
repo: https://charts.jetstack.io
targetNamespace: cert-manager
|
|
For a single node cluster it would seem sufficient to enable the built in letsencrypt integration that Traefik has and saving the cert in a persistent volume. This way no additional components are required. Could this be something the install script optionally does? It looks like the helm chart used supports this. |
|
I'm using Traefik for ssl termination on my single node cluster and I indeed found it much simpler than
traefik:
rbac:
enabled: true
dashboard:
enabled: true
domain: "traefik.example.com"
ssl:
enabled: true
acme:
logging: true
enabled: true
email: "ljani@example.com"
challengeType: dns-01
staging: true
dnsProvider:
name: duckdns
duckdns:
DUCKDNS_TOKEN: 123
domains:
enabled: true
domainsList:
- main: "traefik.example.com"
- sans:
- "otherthing.example.com"
persistence:
enabled: true
storageClass: my-traefik-acme
otherthing:
ingress:
annotations:
kubernetes.io/ingress.class: traefik
hosts:
- name: otherthing.example.com
|
|
I'm a newbie with Kubernetes. @ljani can you please give a step by step, or more detailed, example of using traefik with let's encrypt on k3s? please |
|
@padiazg What else information do you need in addition to the steps above? Here's how to define that |
|
+1 @padiazg |
|
I too struggled with this as a k3s newbie. I finally managed to get it going and of course in hindsight it's pretty simple :-) I blogged about the setup here: https://pascalw.me/blog/2019/07/02/k3s-https-letsencrypt.html. |
|
But still: the following allows me to install cert-manager on k8s but fails on k3s (docker-compose, v0.7.0): No matter how long I wait the APIService never becomes available although the deployments and the service are available. The APIService cannot reach the UPDATE: There is a corresponding cert-manager issue. However installing cert-manager without webhook works: Alternatively a working installation using kustomize with a helm plugin can be found here. |
|
hi guys, For compliance reasons, i have to use certificates issued by a CA - both wildcard and single domain. I cannot use letsencrypt. What is the k3s configuration i need for the inbuilt traefik to pick these certificates and then route the domains to the right pods ? Another recommendation i have heard is to not terminate https at the traefik ingress and instead run a second nginx/haproxy inside which actually does the termination. Not sure if this is ideal, but if no other way is possible.. ill go this way. The SSL stuff is more important than performance. Not sure if i should file a fresh bug...but i have been stuck on this for a long time and am just not able to figure it out. Any help would be much appreciated. |
|
Hi @sandys k3s has a secret that is used by treafik named: After you updated the secret you need to delete the traefik pod. After that it will work. For routing you need an ingress controller, this is an example: Hope gives you a helping hand into the right direction. If you need more help, just ask and I'll try to answer your question. |
|
Using the information in issue #276, I was able to get ACME certificates working without the above mentioned |
|
We're moving away from Traefik and will be using Nginx as our default ingress controller moving forward (our upcoming v1.17.4 release should use Nginx by default for new installs). See #817 for details but if you have any concerns/complaints feel free to list them here in this issue. You can still use Traefik if you want, it just won't be the default in new installs. I would like to close this issue soon since Traefik will no longer be the default ingress controller going forward. |
|
Thanks for the update! Will there be preconfigured letsencrypt support with the new ingress setup? |
|
@lpil I don't know if I can easily provide an answer. I know Traefik supports LE by default but I do not know much about Nginx and LE. If they support it then there's no reason I can see we would not as well. |
|
Typically one would include certbot or similar which would handle provisioning the cert for nginx. Are there plans to include it with the new nginx ingress? |
|
Traefik is still here for the next version 1.17.4
|
|
Since traefik comes installed with k3s by default, why do we need cert-manager? Can't we just use traefik for Lets Encrypt and not install cert-manager? |
But how? I'm on a fight with k3s, rancher to deploy my services. By the way, the Rancher Admin uses cert-manager |
Like this? https://community.hetzner.com/tutorials/howto-k8s-traefik-certmanager |
|
@kidproquo using cert-manager you can decouple your applications from the Ingress controller implementation (traefik in case of k3s) allowing them to work with other Ingress controllers as well. Also traefik can only manage certificates for your Ingresses but there are other use cases where you need certificates (e.g. for a kubernetes APIService or webhook) which cert-manager can manage for you as well. |
|
If you have several instances of traefik in your cluster and you are not using traefik EE (the paid version) you have to use cert-manager if you want LE cert management. For single traefik instance per cluster, traefik without the cert manager may be adequate in managing the LE certs. |
|
And this is just to report that I was able to successfully configure Cert Manager and it seems to work fine with Traefik2 which comes with v1.21.0+k3s1. Nothing special were required, just followed the documentation. |
|
somehow cert-manager 1.5.4 doesn't work for me but 1.5.3 still works like a charm on a fresh k3s setup with traefik. /edit /edit2 apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email: mailuser@mailserver.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: le-staging-issuer-account-key
solvers:
- http01:
ingress:
ingressTemplate:
metadata:
annotations:
kubernetes.io/ingress.class: traefik |
|
The problem seems to be that older versions of cert-manager "translated" the Out of the box, traefik only understands the annotation and not newer version. One workaround would be the comment above mine. Another workaround is to apply another small resource to your cluster: apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: traefik
spec:
controller: traefik.io/ingress-controllerSimply apply this with kubectl and then the "old" ClusterIssuer version should work without any problem: apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email: mailuser@mailserver.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: le-staging-issuer-account-key
solvers:
- http01:
ingress:
class: traefik # this has to match the IngressClass metadata.name above. |
|
traefik 自动签发证书及可视化面板 https://www.rehiy.com/post/392 |
|
Closing this issue as Stale. |
and others
Is your feature request related to a problem? Please describe.
I can't seem to find a way to get existing certs into the container, or to allow LetsEncrypt certificates to survive pod termination, other than writing my own Traefik deployment to add a PersistentVolume or deploying Consul alongside. It would be neat to have documentation on the "proper" way of doing this (I assume there is one, and I'm just not knowledgeable enough about Kubernetes to find it).
Describe the solution you'd like
Documentation covering HTTPS with the built-in Traefik, preferably with existing certificates and with LetsEncrypt.
Describe alternatives you've considered
I could disable the built-in Traefik and roll my own, or run Consul alongside, but both seem like a lot of effort for something that feels like a base requirement in a great many use cases.
Additional context
None
The text was updated successfully, but these errors were encountered: