Skip to content

Commit

Permalink
multicluster: Use the proxy as an HTTP gateway (#4528)
Browse files Browse the repository at this point in the history 
This change modifies the linkerd-gateway component to use the inbound
proxy, rather than nginx, for gateway. This allows us to detect loops and
propagate identity through the gateway.

This change also cleans up port naming to `mc-gateway` and `mc-probe`
to resolve conflicts with Kubernetes validation.

---

* proxy: v2.99.0

The proxy can now operate as gateway, routing requests from its inbound
proxy to the outbound proxy, without passing the requests to a local
application. This supports Linkerd's multicluster feature by adding a
`Forwarded` header to propagate the original client identity and assist
in loop detection.

---

* Add loop detection to inbound & TCP forwarding (linkerd/linkerd2-proxy#527)
* Test loop detection (linkerd/linkerd2-proxy#532)
* fallback: Unwrap errors recursively (linkerd/linkerd2-proxy#534)
* app: Split inbound/outbound constructors into components (linkerd/linkerd2-proxy#533)
* Introduce a gateway between inbound and outbound (linkerd/linkerd2-proxy#540)
* gateway: Add a Forwarded header (linkerd/linkerd2-proxy#544)
* gateway: Return errors instead of responses (linkerd/linkerd2-proxy#547)
* Fail requests that loop through the gateway (linkerd/linkerd2-proxy#545)

* inject: Support config.linkerd.io/enable-gateway

This change introduces a new annotation,
config.linkerd.io/enable-gateway, that, when set, enables the proxy to
act as a gateway, routing all traffic targetting the inbound listener
through the outbound proxy.

This also removes the nginx default listener and gateway port of 4180,
instead using 4143 (the inbound port).

* proxy: v2.100.0

This change modifies the inbound gateway caching so that requests may be
routed to multiple leaves of a traffic split.

---

* inbound: Do not cache gateway services (linkerd/linkerd2-proxy#549)
  • Loading branch information
@olix0r
olix0r committed Jun 3, 2020
1 parent 903fb0f commit 7cc5e5c
Show file tree
Hide file tree
Showing 13 changed files with 320 additions and 306 deletions.
2 changes: 1 addition & 1 deletion .proxy-version
View file
@@ -1 +1 @@
v2.98.0
v2.100.0
2 changes: 1 addition & 1 deletion charts/linkerd2-multicluster/README.md
View file
Expand Up @@ -22,7 +22,7 @@ linkerd2-multicluster chart and their default values.
|`gatewayName` | The name of the gateway that will be installed |`linkerd-gateway` |
|`gatewayNginxImage` | The Nginx image |`nginx` |
|`gatewayNginxImageVersion` | The version of the Nginx image |`1.17` |
|`gatewayPort` | The port on which all the gateway will accept incoming traffic |`4180` |
|`gatewayPort` | The port on which all the gateway will accept incoming traffic |`4143` |
|`gatewayProbePath` | The path that will be used by remote clusters for determining whether the gateway is alive |`/health` |
|`gatewayProbePort` | The port used for liveliness probing |`4181` |
|`gatewayProbeSeconds` | The interval (in seconds) between liveness probes |`3` |
Expand Down
25 changes: 13 additions & 12 deletions charts/linkerd2-multicluster/templates/gateway.yaml
View file
Expand Up @@ -11,12 +11,12 @@ data:
nginx.conf: |-
events {
}
stream {
server {
listen {{.Values.gatewayPort}};
proxy_pass 127.0.0.1:{{.Values.proxyOutboundPort}};
}
}
stream {
server {
listen 4180;
proxy_pass 127.0.0.1:{{.Values.proxyOutboundPort}};
}
}
http {
server {
listen {{.Values.gatewayProbePort}};
Expand Down Expand Up @@ -56,7 +56,8 @@ spec:
annotations:
{{.Values.createdByAnnotation}}: {{default (printf "linkerd/helm %s" .Values.linkerdVersion) .Values.cliVersion}}
linkerd.io/inject: enabled
config.linkerd.io/proxy-require-identity-inbound-ports: "{{.Values.gatewayProbePort}},{{.Values.gatewayPort}}"
config.linkerd.io/proxy-require-identity-inbound-ports: "{{.Values.gatewayProbePort}},{{.Values.gatewayPort}},4180"
config.linkerd.io/enable-gateway: "true"
labels:
app: {{.Values.gatewayName}}
spec:
Expand All @@ -78,9 +79,9 @@ spec:
initialDelaySeconds: 10
image: {{.Values.gatewayNginxImage}}:{{.Values.gatewayNginxImageVersion}}
ports:
- name: linkerd-gateway
containerPort: {{.Values.gatewayPort}}
- name: linkerd-gateway-probe
- name: mc-relay
containerPort: 4180
- name: mc-probe
containerPort: {{.Values.gatewayProbePort}}
- name: local-probe
containerPort: {{.Values.gatewayLocalProbePort}}
Expand All @@ -102,10 +103,10 @@ metadata:
{{.Values.createdByAnnotation}}: {{default (printf "linkerd/helm %s" .Values.linkerdVersion) .Values.cliVersion}}
spec:
ports:
- name: linkerd-gateway
- name: mc-gateway
port: {{.Values.gatewayPort}}
protocol: TCP
- name: linkerd-gateway-probe
- name: mc-probe
port: {{.Values.gatewayProbePort}}
protocol: TCP
selector:
Expand Down
2 changes: 1 addition & 1 deletion charts/linkerd2-multicluster/values.yaml
View file
Expand Up @@ -8,7 +8,7 @@ gatewayLocalProbePort: 8888
gatewayName: linkerd-gateway
gatewayNginxImage: nginx
gatewayNginxImageVersion: 1.17
gatewayPort: 4180
gatewayPort: 4143
gatewayProbePath: /health
gatewayProbePort: 4181
gatewayProbeSeconds: 3
Expand Down
11 changes: 7 additions & 4 deletions charts/partials/templates/_proxy.tpl
View file
Expand Up @@ -18,12 +18,15 @@ env:
value: 127.0.0.1:{{.Values.global.proxy.ports.outbound}}
- name: LINKERD2_PROXY_INBOUND_LISTEN_ADDR
value: 0.0.0.0:{{.Values.global.proxy.ports.inbound}}
{{ if .Values.global.proxy.isGateway -}}
- name: LINKERD2_PROXY_INBOUND_GATEWAY_SUFFIXES
value: {{printf "svc.%s." .Values.global.clusterDomain}}
{{ end -}}
- name: LINKERD2_PROXY_DESTINATION_GET_SUFFIXES
{{- $internalProfileSuffix := printf "svc.%s." .Values.global.clusterDomain }}
value: {{ternary "." $internalProfileSuffix .Values.global.proxy.enableExternalProfiles}}
value: {{printf "svc.%s." .Values.global.clusterDomain}}
- name: LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES
{{- $internalProfileSuffix := printf "svc.%s." .Values.global.clusterDomain }}
value: {{ternary "." $internalProfileSuffix .Values.global.proxy.enableExternalProfiles}}
{{- $internalDomain := printf "svc.%s." .Values.global.clusterDomain }}
value: {{ternary "." $internalDomain .Values.global.proxy.enableExternalProfiles}}
- name: LINKERD2_PROXY_INBOUND_ACCEPT_KEEPALIVE
value: 10000ms
- name: LINKERD2_PROXY_OUTBOUND_CONNECT_KEEPALIVE
Expand Down
8 changes: 4 additions & 4 deletions controller/cmd/service-mirror/cluster_watcher_test_util.go
View file
Expand Up @@ -104,7 +104,7 @@ var createServiceWrongGatewaySpec = &testEnvironment{
},
},
remoteResources: []string{
gatewayAsYaml("existing-gateway", "existing-namespace", "222", "192.0.2.127", "linkerd-gateway-wrong", 888, "", 111, "/path", 666),
gatewayAsYaml("existing-gateway", "existing-namespace", "222", "192.0.2.127", "mc-wrong", 888, "", 111, "/path", 666),
},
}

Expand All @@ -130,7 +130,7 @@ var createServiceOkeGatewaySpec = &testEnvironment{
},
},
remoteResources: []string{
gatewayAsYaml("existing-gateway", "existing-namespace", "222", "192.0.2.127", "linkerd-gateway", 888, "gateway-identity", defaultProbePort, defaultProbePath, defaultProbePeriod),
gatewayAsYaml("existing-gateway", "existing-namespace", "222", "192.0.2.127", "mc-gateway", 888, "gateway-identity", defaultProbePort, defaultProbePath, defaultProbePeriod),
},
}

Expand Down Expand Up @@ -193,7 +193,7 @@ var updateServiceToNewGateway = &testEnvironment{
},
},
remoteResources: []string{
gatewayAsYaml("gateway-new", "gateway-ns", "currentGatewayResVersion", "0.0.0.0", "linkerd-gateway", 999, "", defaultProbePort, defaultProbePath, defaultProbePeriod),
gatewayAsYaml("gateway-new", "gateway-ns", "currentGatewayResVersion", "0.0.0.0", "mc-gateway", 999, "", defaultProbePort, defaultProbePath, defaultProbePeriod),
},
localResources: []string{
mirroredServiceAsYaml("test-service-remote", "test-namespace", "gateway", "gateway-ns", "past", "pastGatewayResVersion", []corev1.ServicePort{
Expand Down Expand Up @@ -269,7 +269,7 @@ var updateServiceWithChangedPorts = &testEnvironment{
},
},
remoteResources: []string{
gatewayAsYaml("gateway", "gateway-ns", "currentGatewayResVersion", "192.0.2.127", "linkerd-gateway", 888, "", defaultProbePort, defaultProbePath, defaultProbePeriod),
gatewayAsYaml("gateway", "gateway-ns", "currentGatewayResVersion", "192.0.2.127", "mc-gateway", 888, "", defaultProbePort, defaultProbePath, defaultProbePeriod),
},
localResources: []string{
mirroredServiceAsYaml("test-service-remote", "test-namespace", "gateway", "gateway-ns", "past", "pastGatewayResVersion", []corev1.ServicePort{
Expand Down
2 changes: 1 addition & 1 deletion controller/proxy-injector/fake/data/inject-sidecar-container-spec.yaml
View file
Expand Up @@ -12,7 +12,7 @@ env:
- name: LINKERD2_PROXY_INBOUND_LISTENER
value: tcp://0.0.0.0:4143
- name: LINKERD2_PROXY_DESTINATION_GET_SUFFIXES
value: .
value: svc.cluster.local.
- name: LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES
value: .
- name: LINKERD2_PROXY_POD_NAMESPACE
Expand Down

0 comments on commit 7cc5e5c

Please sign in to comment.