Intermittent connectivity for ingress-nginx with a LoadBalancer service

[dkeightley]

What kind of request is this (question/bug/enhancement/feature request): bug

Steps to reproduce (least amount of steps as possible):

Create a LoadBalancer service in the ingress-nginx namespace with a selector for app: ingress-nginx. In this case an NLB was created for an RKE cluster with the AWS cloudprovider enabled, with annotation:

service.beta.kubernetes.io/aws-load-balancer-type: nlb

In theory the issue should be reproducible with a NodePort service.

Result:

Connectivity and health checks fail intermittently

Other details that may be helpful:

Removing hostNetwork: true resolves the issue, the LoadBalancer service is used to provide external access to ingress-nginx pods.

Cluster information

• Cluster type (Hosted/Infrastructure Provider/Custom/Imported): Infrastructure/EC2
• Kubernetes version (use kubectl version): v1.17.9, rancher/nginx-ingress-controller:nginx-0.32.0-rancher1

gzrancher/rancher#11686

gzrancher/rancher#9159

gzrancher/rancher#7291

[kinarashah]

Available to test in v2.4.9-rc2

[bmdepesa]

reproduced in v2.4.8

• RKE cluster w/ aws cloud provider
• Create a load balancer for ingress-nginx-controller
• Deploy a workload and access through the load balancer over time
• See health checks fail and some connectivity issues

rancher/rancher:v2.4-4095-head

• When configuring the cluster, set:

ingress:
  provider: nginx
  network_mode: none
  extra_args:
     http-port: 8080
     https-port: 8443

• Same scenario as above
• See hostNetwork: true removed from the ingress-nginx-controller DS
• Container ports are bound correctly
• Health checks are passing / no connectivity issues

Still to be tested in v2.5-head / master-head

[bmdepesa]

Tested in:

• rancher/rancher:v2.5-head bebfe0d
• rancher/rancher:master-head 55fc963