HTTP request failed after ISTIO_META_DNS_AUTO_ALLOCATE enabled

[fatedier]

Bug Description

There is a ServiceEntry for example.com, port 9444 as TCP protocol.

HTTP requests to example.com:80 will also succeed.

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: external-auto
spec:
  hosts:
  - example.com
  ports:
  - name: tcp
    number: 9444
    protocol: TCP
  resolution: DNS

After I enable ISTIO_META_DNS_AUTO_ALLOCATE, HTTP requests to example.com:80 always failed. Got 503 connection failure

Istio allocs a new IP for example.com. Destination of connections to this host will be changed to this IP and passthrough cluster will use ORIGIN DST to connect and fail.

Version

client version: 1.12.6
control plane version: 1.12.6
data plane version: 1.12.6

Additional Information

No response

[hzxuzhonghu]

same as #29224

[fatedier]

@hzxuzhonghu Hi，there maybe a little difference with #29224.

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: external-auto
spec:
  hosts:
  - example.com
  ports:
  - name: tcp
    number: 9444
    protocol: TCP
  resolution: DNS

I define a TCP port in this ServiceEntry. Resolve example.com to an IP auto alloced by istio is expected since I will add VirtualService to this host and port. It needs to dstinguish the different host by IP.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: auto-tls
spec:
  hosts:
  - "example.com"
  gateways:
  - mesh
  tcp:
  - match:
    - port: 9444
    route:
    - destination:
        host: sni-proxy
        subset: example-com-9444
        port:
          number: 443

It works well.

But curl example.com:80 failed. I'm not sure if we can route traffics to the alloced IP and ports except 9444 to a STRICT_DNS type cluster.

[howardjohn]

This is expected IMO. Original_dst will not work with this mode, as noted. When you define your svc entry for 9444, you are saying its ONLY on port 9444 (not 80). So you hit passthrough which ends up breaking.

Its not possible for us to somehow only return the fake IP for port 9444 since DNS is not port-aware.

ISTIO_META_DNS_AUTO_ALLOCATE is not something that can ever be 100% reliable

[fatedier]

This is expected IMO. Original_dst will not work with this mode, as noted. When you define your svc entry for 9444, you are saying its ONLY on port 9444 (not 80). So you hit passthrough which ends up breaking.

Its not possible for us to somehow only return the fake IP for port 9444 since DNS is not port-aware.

ISTIO_META_DNS_AUTO_ALLOCATE is not something that can ever be 100% reliable

@howardjohn Alright.

Another question. ISTIO_META_DNS_AUTO_ALLOCATE is global. Can I only enable it for specified serviceentries? So it will not affect existing serviceentries.

[hzxuzhonghu]

There is no such way now

[howardjohn]

If you want the opposite (opt-in to allocate) you can basically DIY it by just setting a random IP in the ServiceEntry.address YAML. That is basically what the feature does.

[fatedier]

@howardjohn Great. Thank you for your reminding.