-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for fetching k8s Ingress TLS data from secrets #2439
Changes from 18 commits
3892fb7
228362c
765fba4
1f96b88
bf648b8
4c0ba9e
18c5f9e
cc0b448
1c60f29
b0a7899
741163e
c13b21a
bce97e9
383b9e8
74616a4
e082818
3ecf770
d0a5c2f
4d156ec
d00111a
6ab8f21
1d2e121
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -333,6 +333,48 @@ echo "$(minikube ip) traefik-ui.minikube" | sudo tee -a /etc/hosts | |
|
||
We should now be able to visit [traefik-ui.minikube](http://traefik-ui.minikube) in the browser and view the Træfik Web UI. | ||
|
||
### Add a TLS Certificate to the Ingress | ||
|
||
!!! note | ||
For this example to work you need a TLS entrypoint. You don't have to provide a TLS certificate at this point. For more details see [here](/configuration/entrypoints/). | ||
|
||
To setup a HTTPS protected ingress, you can leverage the TLS feature of the ingress resource. | ||
|
||
```yaml | ||
apiVersion: extensions/v1beta1 | ||
kind: Ingress | ||
metadata: | ||
name: traefik-web-ui | ||
namespace: kube-system | ||
annotations: | ||
kubernetes.io/ingress.class: traefik | ||
spec: | ||
rules: | ||
- host: traefik-ui.minikube | ||
http: | ||
paths: | ||
- backend: | ||
serviceName: traefik-web-ui | ||
servicePort: 80 | ||
tls: | ||
- hosts: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, I don't see us parsing the hosts list in our code explicitly. Am I missing something? |
||
- traefik-ui.minikube | ||
secretName: traefik-ui-tls-cert | ||
``` | ||
|
||
In addition to the modified ingress you need to provide the TLS certificate via a kubernetes secret in the same namespace as the ingress. The following two commands will generate a new certificate and create a secret containing the key and cert files. | ||
|
||
```shell | ||
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /tmp/tls.key -out /tmp/tls.crt -subj "/CN=traefik-ui.minikube" | ||
kubectl -n kube-system create secret tls traefik-ui-tls-cert --key=/tmp/tls.key --cert=/tmp/tls.crt | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd drop the |
||
``` | ||
|
||
!!! note | ||
The secret must have two entries named `tls.key`and `tls.crt`. See the [kubernetes documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls) for more details. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Excellent extension to the guide 👍 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I had no idea how to do something like this but I'm glad it is OK |
||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you please add a note to explain that the certificates contained into the K8s secrets will be added to all the TLS entryPoints linked to the services which use them (if the label There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would the following ok with you? The TLS certificates will be added to all entrypoints defined by the ingress annotation Did you mean with 'note' the note formatting or just an addition to the documentation? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes the sentence LGTM 👍 |
||
!!! note | ||
The TLS certificates will be added to all entrypoints defined by the ingress annotation `traefik.frontend.entryPoints`. If no such annotation is provided, the TLS certificates will be added to all TLS enabled `defaultEntryPoints`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. s/to all TLS enabled/to all TLS-enabled/ |
||
|
||
## Basic Authentication | ||
|
||
It's possible to add additional authentication annotations in the Ingress rule. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -92,6 +92,23 @@ func iBackend(name string, port intstr.IntOrString) func(*v1beta1.HTTPIngressPat | |
} | ||
} | ||
|
||
func iTLSs(opts ...func(*v1beta1.IngressTLS)) func(*v1beta1.Ingress) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we rename this to |
||
return func(i *v1beta1.Ingress) { | ||
for _, opt := range opts { | ||
iTLS := v1beta1.IngressTLS{} | ||
opt(&iTLS) | ||
i.Spec.TLS = append(i.Spec.TLS, iTLS) | ||
} | ||
} | ||
} | ||
|
||
func iTLS(secret string, hosts ...string) func(*v1beta1.IngressTLS) { | ||
return func(i *v1beta1.IngressTLS) { | ||
i.SecretName = secret | ||
i.Hosts = hosts | ||
} | ||
} | ||
|
||
// Test | ||
|
||
func TestBuildIngress(t *testing.T) { | ||
|
@@ -107,7 +124,11 @@ func TestBuildIngress(t *testing.T) { | |
onePath(iBackend("service2", intstr.FromInt(802))), | ||
), | ||
), | ||
)) | ||
), | ||
iTLSs( | ||
iTLS("tls-secret", "foo"), | ||
), | ||
) | ||
|
||
assert.EqualValues(t, sampleIngress(), i) | ||
} | ||
|
@@ -164,6 +185,12 @@ func sampleIngress() *v1beta1.Ingress { | |
}, | ||
}, | ||
}, | ||
TLS: []v1beta1.IngressTLS{ | ||
{ | ||
Hosts: []string{"foo"}, | ||
SecretName: "tls-secret", | ||
}, | ||
}, | ||
}, | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,6 +19,7 @@ import ( | |
"github.com/containous/traefik/provider" | ||
"github.com/containous/traefik/provider/label" | ||
"github.com/containous/traefik/safe" | ||
"github.com/containous/traefik/tls" | ||
"github.com/containous/traefik/types" | ||
"k8s.io/client-go/pkg/api/v1" | ||
"k8s.io/client-go/pkg/apis/extensions/v1beta1" | ||
|
@@ -354,6 +355,13 @@ func (p *Provider) loadIngresses(k8sClient Client) (*types.Configuration, error) | |
} | ||
} | ||
} | ||
|
||
tlsConfigs, err := getTLSConfigurations(i, k8sClient) | ||
if err != nil { | ||
log.Errorf("Error configuring TLS for ingress %s/%s: %v", i.Namespace, i.Name, err) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I couldn't figure out, if an entry point is TLS-enabled at this point as the default configuration (passing nil) is handled further down the road and override entry points are only available as strings without any context. Therefore, I can't check if an entry point is TLS-enabled or not. But I can't imagine a way to solve this problem properly as either the internals need to know about a TLS configuration error and abort in the case of an HTTPS-only entry point or the provider needs to know about the entry points (default and override). I'm sorry for not pointing this out earlier. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for doing the research. If we can't figure out whether it's safe to accept an illegal definition or not, then my opinion is that we should err on the side of safety and skip it. Better be safe than sorry. What do you (and @dtomcej) think? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Staying on the safe side should be our first choice as this is security critical. I'd also add a note to the docs regarding this and also the deviation from the kubernetes spec (host names are ignored and therefore the wildcard behaviour cannot be implemented). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you refresh my mind again: what's the reason we cannot comply with the Kubernetes spec? Is there a potential path (possibly in the future) we could improve on this point? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In a kubernetes ingress resource multiple TLSes per ingress are allowed. Each consists of two parts: a list of host names and a reference to a kubernetes secret. There are two cases in which a TLS configuration can be:
I think there is a path to implement this feature in the future as this behavior is a special case of the already implemented one in the traefik internals. (https://github.com/containous/traefik/blob/c446c291d9dc49895e69c91e51a054778c71f21a/tls/certificate.go#L150-L154 would be the default behavior, but it can be overwritten by the provider) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Honestly, after doing a bit of research, there does not seem to be any implementation outside of ingress controller annotations to handle the failures of any of the components of an ingress. It appears that it is all-or-nothing, where the tls section of the spec is parsed in parallel with the rules section of the spec, leaving the annotations to handle the relationship between http/rules and tls sections. Therefore, I would now be more inclined to just fail/skip loading the entire ingress on an error loading any portion of the tls section of the spec, since there is no way (without going into all the middlewares) to know how important the separate tls sections are. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @gopenguin thanks for the clarification with regards to the wildcard matter. As long as we clearly document the constraints, I'm fine with it. In the light of @dtomcej's latest comment: could you please change the logic so that we skip illegal Ingress definitions? AFAICS, that is the last point that keeps us from merging this PR. |
||
} else { | ||
templateObjects.TLSConfiguration = append(templateObjects.TLSConfiguration, tlsConfigs...) | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we put the TLS logic into a dedicated function? |
||
} | ||
return &templateObjects, nil | ||
} | ||
|
@@ -441,6 +449,48 @@ func loadAuthCredentials(namespace, secretName string, k8sClient Client) ([]stri | |
return creds, nil | ||
} | ||
|
||
func getTLSConfigurations(ingress *v1beta1.Ingress, k8sClient Client) ([]*tls.Configuration, error) { | ||
var tlsConfigs []*tls.Configuration | ||
|
||
for _, t := range ingress.Spec.TLS { | ||
tlsSecret, exists, err := k8sClient.GetSecret(ingress.Namespace, t.SecretName) | ||
if err != nil { | ||
return nil, fmt.Errorf("failed to fetch secret %s/%s: %v", ingress.Namespace, t.SecretName, err) | ||
} | ||
if !exists { | ||
return nil, fmt.Errorf("secret %s/%s does not exist", ingress.Namespace, t.SecretName) | ||
} | ||
|
||
tlsCrtData, tlsCrtExists := tlsSecret.Data["tls.crt"] | ||
tlsKeyData, tlsKeyExists := tlsSecret.Data["tls.key"] | ||
|
||
var missingEntries []string | ||
if !tlsCrtExists { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. WDYT about testing if both There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That sounds reasonable. Would be There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My suggestion would be |
||
missingEntries = append(missingEntries, "tls.crt") | ||
} | ||
if !tlsKeyExists { | ||
missingEntries = append(missingEntries, "tls.key") | ||
} | ||
if len(missingEntries) > 0 { | ||
return nil, fmt.Errorf("secret %s/%s is missing the following TLS data entries: %s", ingress.Namespace, t.SecretName, strings.Join(missingEntries, ", ")) | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like this a lot 👍 |
||
|
||
entryPoints := label.GetSliceStringValue(ingress.Annotations, label.TraefikFrontendEntryPoints) | ||
|
||
tlsConfig := &tls.Configuration{ | ||
EntryPoints: entryPoints, | ||
Certificate: &tls.Certificate{ | ||
CertFile: tls.FileOrContent(tlsCrtData), | ||
KeyFile: tls.FileOrContent(tlsKeyData), | ||
}, | ||
} | ||
|
||
tlsConfigs = append(tlsConfigs, tlsConfig) | ||
} | ||
|
||
return tlsConfigs, nil | ||
} | ||
|
||
func endpointPortNumber(servicePort v1.ServicePort, endpointPorts []v1.EndpointPort) int { | ||
if len(endpointPorts) > 0 { | ||
//name is optional if there is only one port | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/a HTTPS protected/an HTTPS-protected/