Mixed case host in VirtualService causes RDS to go STALE.

[andy-townsend]

Is this the right place to submit this?

• This is not a security vulnerability or a crashing bug
• This is not a question about how to use Istio

Bug Description

Last week we ran into an issue where a duplicate VirtualService entry was added to one of our clusters and it causes the RDS status to go STALE. I've been able to duplicate the issue and its down to where there's mixed case in the hosts entry. eg:

  hosts:
  - test-namespace-HELLO.int.lab-ie-01.mydomain.uk

and

  hosts:
  - test-namespace-hello.int.lab-ie-01.mydomain.uk

We get the following error in the logs

{"level":"warning","time":"2024-02-29T17:19:42.202215Z","scope":"envoy config","msg":"gRPC config for type.googleapis.com/envoy.config.route.v3.RouteConfiguration rejected: Only unique values for domains are permitted. Duplicate entry of domain test-namespace-hello.int.lab-ie-01.mydomain.uk in route http.29000","caller":"external/envoy/source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138","thread":15}

I can replicate the issue by applying this yaml to the cluster.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: test-duplicate-vs-1
  namespace: test-namespace
spec:
  gateways:
  - ingress-private/private-cluster
  - ingress-private/private-env
  hosts:
  - test-namespace-HELLO.int.lab-ie-01.mydomain.uk
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: hello-world.test-namespace.svc.cluster.local
        port:
          number: 8080
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: test-duplicate-vs-2
  namespace: test-namespace
spec:
  gateways:
  - ingress-private/private-cluster
  - ingress-private/private-env
  hosts:
  - test-namespace-hello.int.lab-ie-01.mydomain.uk
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: hello-world.test-namespace.svc.cluster.local
        port:
          number: 8080

We found that this broke routing within the cluster for this service which started generating errors. I can see there's this issue where DNS names were allowed to be case-insensitive and also this issue that prevents duplicate routes (but doesn't account for case?).

Version

$ istioctl version 
client version: 1.20.3
control plane version: 1.20.3
data plane version: 1.20.3 (32 proxies)

$ kubectl version
Client Version: v1.28.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.5-eks-5e0fdde

Additional Information

No response

[howardjohn]

cc @ramaraochavali

[hzxuzhonghu]

We should ignore case in the domain

[hzxuzhonghu]

https://www.rfc-editor.org/rfc/rfc7230#section-3.2 all headers are case-insenstive

[howardjohn]

https://www.rfc-editor.org/rfc/rfc7230#section-3.2 all headers are case-insenstive

...case-insensitive field name..

Just the header name is, this is about the header value. But Host specifically is case insensitive so I agree we should ignore the case here

[ramaraochavali]

Can you please share gateways as well? I guess you have similar duplicate host name in gateways? We have fixed this for sidecar but not gateways.

[andy-townsend]

For the gateway config we're using a wildcard like this;

---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  annotations:
    meta.helm.sh/release-name: private-cluster
    meta.helm.sh/release-namespace: ingress-private
  name: private-cluster
  namespace: ingress-private
spec:
  selector:
    app: istio-ingressgateway
  servers:
  - hosts:
    - int.lab-ie-01.mydomain.uk
    - '*.int.lab-ie-01.mydomain.uk'
    port:
      name: http
      number: 29000
      protocol: HTTP
    tls:
      httpsRedirect: true
  - hosts:
    - int.lab-ie-01.mydomain.uk
    - '*.int.lab-ie-01.mydomain.uk'
    port:
      name: https
      number: 29001
      protocol: HTTPS
    tls:
      cipherSuites:
      - ECDHE-RSA-CHACHA20-POLY1305
      - ECDHE-RSA-AES256-GCM-SHA384
      - ECDHE-RSA-AES256-SHA
      credentialName: private-cluster-tls
      maxProtocolVersion: TLSV1_3
      minProtocolVersion: TLSV1_2
      mode: SIMPLE

[andy-townsend]

@ramaraochavali - I see this is closed as fixed by #49674 but do you know when that will be released?

[hzxuzhonghu]

In 1.22，and need to backport