NextGenRoute Controller to support TLS passthrough without SNI in client hello

[leonseng]

Title

NextGenRoute Controller to support TLS passthrough without SNI in client hello

Description

When an OpenShift Route is configured with spec.tls.termination: passthrough, NextGenRoute Controller creates a VS with an iRule that load balances traffic to different pools based on the SNI presented in the TLS Client Hello message. The VS does NOT have a default pool defined.

For clients which do not provide SNI in Client Hello, no pool can be selected, hence traffic is not passed to the pods in the OpenShift cluster.

Actual Problem

For applications hosting their own TLS certificates within the OpenShift clusters, clients which do not provide SNI in Client Hello are unable to access said applications in TLS passthrough mode.

Solution Proposed

Some toggle in the Route manifest to configure VS with pool member set to the backend pods without the need to select pools based on SNI in the iRules.

Alternatives

One suggestion is to remove the existing iRule with a Policy CR, but without a default pool defined, BIG-IP does not know where to send traffic to.

Maybe a way to configure default pool on the VS will be good.

Additional context

CIS Helm values

bigip_login_secret: bigip-login
rbac:
  create: true
serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: k8s-bigip-ctlr
# This namespace is where the Controller lives;
namespace: kube-system
args:
  # See http://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest/#controller-configuration-parameters
  # REQUIRED Params
  bigip_url: 10.1.1.5
  insecure: True
  pool_member_type: nodeport
  bigip_partition: OpenShift
  log_as3_response: True
  namespaces: ["kube-system", "app"]
  controller-mode: openshift
  extended-spec-configmap: kube-system/cis-extended-spec
image:
  # Use the tag to target a specific version of the Controller
  user: f5networks
  repo: k8s-bigip-ctlr
  pullPolicy: Always
resources: {}
version: latest

Extended Route spec

apiVersion: v1
data:
  extendedSpec: |
    extendedRouteSpec:
    - namespace: app
      vserverAddr: 10.1.10.200
      vserverName: override-vs
      policyCR: app/my-irules
kind: ConfigMap
metadata:
  labels:
    f5nr: "true"
  name: cis-extended-spec-override
  namespace: app

Policy to remove iRule from the VS

apiVersion: cis.f5.com/v1
kind: Policy
metadata:
  annotations:
  labels:
    f5cr: "true"
  name: my-irules
  namespace: app
spec:
  iRules:
    secure: none

[trinaths]

@leonseng We support defaultPool with VS CRD see https://github.com/F5Networks/k8s-bigip-ctlr/tree/2.x-master/docs/config_examples/customResource/VirtualServer/defaultpool

[leonseng]

The user prefers to use OpenShift Routes, hence the ask to support it in NextGenRoute controller.

[trinaths]

Created [CONTCNTR-4579] for internal tracking.

[lavanya-f5]

@leonseng please test with dev build quay.io/f5networks/k8s-bigip-ctlr-devel:a9048f18b96e92e5f0d024c76cebe20752339c19 and provide your feedback

latest crd schema to apply - https://github.com/F5Networks/k8s-bigip-ctlr/blob/2.x-master/docs/config_examples/customResourceDefinitions/incubator/customresourcedefinitions.yml

set defaultPool via policy
https://github.com/F5Networks/k8s-bigip-ctlr/blob/2.x-master/docs/config_examples/customResource/Policy/policy-with-default-pool.yaml

[leonseng]

looks promising in my initial test. Let me run it past the end user to confirm

This is my test config

$ k -n app get cm cis-extended-spec-override -o yaml
apiVersion: v1
data:
  extendedSpec: |
    extendedRouteSpec:
    - namespace: app
      vserverAddr: 10.1.10.200
      vserverName: override-vs
      policyCR: app/tls-passthrough-no-sni
kind: ConfigMap
metadata:
  labels:
    f5nr: "true"
  name: cis-extended-spec-override
  namespace: app
  resourceVersion: "4192547"
  uid: d8440f40-cf37-4a5c-a464-274ee0ad3fc3

$ k -n app get policy tls-passthrough-no-sni -o yaml
apiVersion: cis.f5.com/v1
kind: Policy
metadata:
  labels:
    f5cr: "true"
  name: tls-passthrough-no-sni
  namespace: app
spec:
  defaultPool:
    reference: service
    service: nginx-service
    servicePort: 8443
  snat: auto
  iRuleList:
  - /Common/ignore-ssl-profile

iRule on BIG-IP /Common/ignore-ssl-profile

when CLIENT_DATA priority 1 {
 	SSL::disable clientside
 	HTTP::disable
}

Resulting in a VS with the SNI iRule, but this time with a default pool

Testing connection against Virtual Server returns server certificate from pod, not from the clientssl profile on the VS as expected

openssl s_client -connect 10.1.10.200:443
CONNECTED(00000003)
depth=0 O = mkcert development certificate, OU = leon@123
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 O = mkcert development certificate, OU = leon@123
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/O=mkcert development certificate/OU=leon@123
   i:/O=mkcert development CA/OU=leon@5NNS0F3/CN=mkcert leon@123
---