Skip to content

Commit

Permalink
Merge pull request #86 from Charliekenney23/feat/nodebal-per-port-config
Browse files Browse the repository at this point in the history 
allow configuring proxy-protocol per port
  • Loading branch information
@0xch4z
0xch4z committed Jan 7, 2021
2 parents 62b517e + 3420b69 commit a0465bd
Show file tree
Hide file tree
Showing 6 changed files with 209 additions and 47 deletions.
29 changes: 20 additions & 9 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ Annotation (Suffix) | Values | Default | Description
---|---|---|---
`throttle` | `0`-`20` (`0` to disable) | `20` | Client Connection Throttle, which limits the number of subsequent new connections per second from the same client IP
`default-protocol` | `tcp`, `http`, `https` | `tcp` | This annotation is used to specify the default protocol for Linode NodeBalancer.
`proxy-protocol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer
`port-*` | json (e.g. `{ "tls-secret-name": "prod-app-tls", "protocol": "https"}`) | | Specifies the secret and protocol for a port corresponding secrets. The secret type should be `kubernetes.io/tls`. `*` is the port being configured, e.g. `linode-loadbalancer-port-443`
`default-proxy-protocol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer.
`port-*` | json (e.g. `{ "tls-secret-name": "prod-app-tls", "protocol": "https", "proxy-protocol": "v2"}`) | | Specifies port specific NodeBalancer configuration. See [Port Specific Configuration](#port-specific-configuration). `*` is the port being configured, e.g. `linode-loadbalancer-port-443`
`check-type` | `none`, `connection`, `http`, `http_body` | | The type of health check to perform against back-ends to ensure they are serving requests
`check-path` | string | | The URL path to check on each back-end during health checks
`check-body` | string | | Text which must be present in the response body to pass the NodeBalancer health check
Expand All @@ -58,17 +58,28 @@ Annotation (Suffix) | Values | Default | Description

#### Deprecated Annotations

These annotations are deprecated, and will be removed Q3 2020.
These annotations are deprecated, and will be removed in a future release.

Annotation (Suffix) | Values | Default | Description
---|---|---|---
`protocol` | `tcp`, `http`, `https` | `tcp` | This annotation is used to specify the default protocol for Linode NodeBalancer. For ports specified in the `linode-loadbalancer-tls-ports` annotation, this protocol is overwritten to `https`
`tls` | json array (e.g. `[ { "tls-secret-name": "prod-app-tls", "port": 443}, {"tls-secret-name": "dev-app-tls", "port": 8443} ]`) | | Specifies TLS ports with their corresponding secrets, the secret type should be `kubernetes.io/tls
Annotation (Suffix) | Values | Default | Description | Scheduled Removal
---|---|---|---|---
`protocol` | `tcp`, `http`, `https` | `tcp` | This annotation is used to specify the default protocol for Linode NodeBalancer. For ports specified in the `linode-loadbalancer-tls-ports` annotation, this protocol is overwritten to `https` | Q4 2020
`proxy-protcol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer | Q4 2021
`tls` | json array (e.g. `[ { "tls-secret-name": "prod-app-tls", "port": 443}, {"tls-secret-name": "dev-app-tls", "port": 8443} ]`) | | Specifies TLS ports with their corresponding secrets, the secret type should be `kubernetes.io/tls | Q4 2020

#### Annotation bool values

For annotations with bool value types, `"1"`, `"t"`, `"T"`, `"True"`, `"true"` and `"True"` are valid string representations of `true`. Any other values will be interpreted as false. For more details, see [strconv.ParseBool](https://golang.org/pkg/strconv/#ParseBool).

#### Port Specific Configuration

These configuration options can be specified via the `port-*` annotation, encoded in JSON.

Key | Values | Default | Description
---|---|---|---
`protocol` | `tcp`, `http`, `https` | `tcp` | Specifies protocol of the NodeBalancer port. Overwrites `default-protocol`.
`proxy-protocol` | `none`, `v1`, `v2` | `none` | Specifies whether to use a version of Proxy Protocol on the underlying NodeBalancer. Overwrites `default-proxy-protocol`.
`tls-secret-name` | string | | Specifies a secret to use for TLS. The secret type should be `kubernetes.io/tls`.

#### Example usage

```yaml
Expand Down Expand Up @@ -128,10 +139,10 @@ spec:

See more in the [examples directory](examples)

## Why `stickiness` and `algorithm` annotations don't exit
## Why `stickiness` and `algorithm` annotations don't exist

As kube-proxy will simply double-hop the traffic to a random backend Pod anyway, it doesn't matter which backend Node traffic is forwarded-to for the sake of session stickiness.
So these annotations are not necessary to implement session stickiness.
These annotations are not necessary to implement session stickiness, as kube-proxy will simply double-hop the packets to a random backend Pod. It would not make a difference to set a backend Node that would receive the network traffic in an attempt to set session stickiness.

## How to use sessionAffinity

Expand Down
48 changes: 30 additions & 18 deletions cloud/linode/loadbalancers.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ import (
const (
// annLinodeDefaultProtocol is the annotation used to specify the default protocol
// for Linode load balancers. Options are tcp, http and https. Defaults to tcp.
annLinodeDefaultProtocol = "service.beta.kubernetes.io/linode-loadbalancer-default-protocol"
annLinodePortConfigPrefix = "service.beta.kubernetes.io/linode-loadbalancer-port-"
annLinodeProxyProtocol = "service.beta.kubernetes.io/linode-loadbalancer-proxy-protocol"
annLinodeDefaultProtocol = "service.beta.kubernetes.io/linode-loadbalancer-default-protocol"
annLinodePortConfigPrefix = "service.beta.kubernetes.io/linode-loadbalancer-port-"
annLinodeDefaultProxyProtocol = "service.beta.kubernetes.io/linode-loadbalancer-default-proxy-protocol"

annLinodeCheckPath = "service.beta.kubernetes.io/linode-loadbalancer-check-path"
annLinodeCheckBody = "service.beta.kubernetes.io/linode-loadbalancer-check-body"
Expand Down Expand Up @@ -68,11 +68,13 @@ type loadbalancers struct {
type portConfigAnnotation struct {
TLSSecretName string `json:"tls-secret-name"`
Protocol string `json:"protocol"`
ProxyProtocol string `json:"proxy-protocol"`
}

type portConfig struct {
TLSSecretName string
Protocol linodego.ConfigProtocol
ProxyProtocol linodego.ConfigProxyProtocol
Port int
}

Expand Down Expand Up @@ -478,9 +480,10 @@ func (l *loadbalancers) buildNodeBalancerConfig(service *v1.Service, port int) (
}

config := linodego.NodeBalancerConfig{
Port: port,
Protocol: portConfig.Protocol,
Check: health,
Port: port,
Protocol: portConfig.Protocol,
ProxyProtocol: portConfig.ProxyProtocol,
Check: health,
}

if health == linodego.CheckHTTP || health == linodego.CheckHTTPBody {
Expand Down Expand Up @@ -530,17 +533,6 @@ func (l *loadbalancers) buildNodeBalancerConfig(service *v1.Service, port int) (
}
config.CheckPassive = checkPassive

proxyProtocol := linodego.ProxyProtocolNone
if pp, ok := service.Annotations[annLinodeProxyProtocol]; ok {
switch linodego.ConfigProxyProtocol(pp) {
case linodego.ProxyProtocolNone, linodego.ProxyProtocolV1, linodego.ProxyProtocolV2:
proxyProtocol = linodego.ConfigProxyProtocol(pp)
default:
return config, fmt.Errorf("invalid NodeBalancer proxy protocol value '%s'", pp)
}
}
config.ProxyProtocol = proxyProtocol

if portConfig.Protocol == linodego.ProtocolHTTPS {
if err = l.addTLSCert(service, &config, portConfig); err != nil {
return config, err
Expand Down Expand Up @@ -643,15 +635,35 @@ func getPortConfig(service *v1.Service, port int) (portConfig, error) {
protocol = "tcp"
}
}

protocol = strings.ToLower(protocol)

proxyProtocol := portConfigAnnotation.ProxyProtocol
if proxyProtocol == "" {
var ok bool
for _, ann := range []string{annLinodeDefaultProxyProtocol, annLinodeProxyProtocol} {
proxyProtocol, ok = service.Annotations[ann]
if ok {
break
} else {
proxyProtocol = string(linodego.ProxyProtocolNone)
}
}
}

if protocol != "tcp" && protocol != "http" && protocol != "https" {
return portConfig, fmt.Errorf("invalid protocol: %q specified", protocol)
}

switch proxyProtocol {
case string(linodego.ProxyProtocolNone), string(linodego.ProxyProtocolV1), string(linodego.ProxyProtocolV2):
break
default:
return portConfig, fmt.Errorf("invalid NodeBalancer proxy protocol value '%s'", proxyProtocol)
}

portConfig.Port = port
portConfig.Protocol = linodego.ConfigProtocol(protocol)
portConfig.ProxyProtocol = linodego.ConfigProxyProtocol(proxyProtocol)
portConfig.TLSSecretName = portConfigAnnotation.TLSSecretName

return portConfig, nil
Expand Down
1 change: 1 addition & 0 deletions cloud/linode/loadbalancers_deprecated.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
const (
annLinodeProtocolDeprecated = "service.beta.kubernetes.io/linode-loadbalancer-protocol"
annLinodeLoadBalancerTLSDeprecated = "service.beta.kubernetes.io/linode-loadbalancer-tls"
annLinodeProxyProtocol = "service.beta.kubernetes.io/linode-loadbalancer-proxy-protocol"
)

type tlsAnnotationDeprecated struct {
Expand Down
66 changes: 60 additions & 6 deletions cloud/linode/loadbalancers_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -573,7 +573,7 @@ func testUpdateLoadBalancerAddProxyProtocol(t *testing.T, client *linodego.Clien

svc.Status.LoadBalancer = *makeLoadBalancerStatus(nodeBalancer)
svc.ObjectMeta.SetAnnotations(map[string]string{
annLinodeProxyProtocol: string(tc.proxyProtocolConfig),
annLinodeDefaultProxyProtocol: string(tc.proxyProtocolConfig),
})

stubService(fakeClientset, svc)
Expand Down Expand Up @@ -771,6 +771,60 @@ func Test_getPortConfig(t *testing.T) {
expectedPortConfig portConfig
err error
}{
{
"default no proxy protocol specified",
&v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: randString(10),
UID: "abc123",
},
},
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolNone},
nil,
},
{
"default proxy protocol specified",
&v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: randString(10),
UID: "abc123",
Annotations: map[string]string{
annLinodeDefaultProxyProtocol: string(linodego.ProxyProtocolV2),
},
},
},
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolV2},
nil,
},
{
"port specific proxy protocol specified",
&v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: randString(10),
UID: "abc123",
Annotations: map[string]string{
annLinodeDefaultProxyProtocol: string(linodego.ProxyProtocolV2),
annLinodePortConfigPrefix + "443": fmt.Sprintf(`{"proxy-protocol": "%s"}`, linodego.ProxyProtocolV1),
},
},
},
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolV1},
nil,
},
{
"default invalid proxy protocol",
&v1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: randString(10),
UID: "abc123",
Annotations: map[string]string{
annLinodeDefaultProxyProtocol: "invalid",
},
},
},
portConfig{},
fmt.Errorf("invalid NodeBalancer proxy protocol value '%s'", "invalid"),
},
{
"default no protocol specified",
&v1.Service{
Expand All @@ -779,7 +833,7 @@ func Test_getPortConfig(t *testing.T) {
UID: "abc123",
},
},
portConfig{Port: 443, Protocol: "tcp"},
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolNone},

nil,
},
Expand All @@ -794,7 +848,7 @@ func Test_getPortConfig(t *testing.T) {
},
},
},
portConfig{Port: 443, Protocol: "tcp"},
portConfig{Port: 443, Protocol: "tcp", ProxyProtocol: linodego.ProxyProtocolNone},
nil,
},
{
Expand All @@ -808,7 +862,7 @@ func Test_getPortConfig(t *testing.T) {
},
},
},
portConfig{Port: 443, Protocol: "http"},
portConfig{Port: 443, Protocol: "http", ProxyProtocol: linodego.ProxyProtocolNone},
nil,
},
{
Expand Down Expand Up @@ -837,7 +891,7 @@ func Test_getPortConfig(t *testing.T) {
},
},
},
portConfig{Port: 443, Protocol: "http"},
portConfig{Port: 443, Protocol: "http", ProxyProtocol: linodego.ProxyProtocolNone},
nil,
},
{
Expand All @@ -851,7 +905,7 @@ func Test_getPortConfig(t *testing.T) {
},
},
},
portConfig{Port: 443, Protocol: "http"},
portConfig{Port: 443, Protocol: "http", ProxyProtocol: linodego.ProxyProtocolNone},
nil,
},
{
Expand Down
Loading

0 comments on commit a0465bd

Please sign in to comment.