istio-proxy 404 when header Host is not specified

[adecchi-2inno]

Bug Description

I am running istio 1.15.2 on GKE 1.24.3. The problem is when I try to get some services via the public or private endpoint.
For example, If i execute the curl command from the container application I get the status code 404, If I execute the same curl command from the istio-proxy sidecar or externally I get the status code 200.
Then, if set the Host header in the request it works from the container application I get 200:

curl -XGET http://10.10.7.5/test/health -H "Host: api-internal.mycompany.com:8080" -I

I was looking for the domain at config_dump and looks ok:

"name": "api-internal.mycompany.com:80", "domains": [ "api-internal.mycompany.com", "api-internal.mycompany.com:80" ],

Version

control plane version: 1.15.2
data plane version: 1.15.2 (15 proxies)

Additional Information

No response

[howardjohn]

The config dump does not look ok :-); it is matching :80 and you are sending :8080 in the host header? But you are actually sending to port 80 so it seems odd to set :8080 in the Host header.

FWIW I think in 1.16 we start ignoring port entirely in the host

[adecchi-2inno]

@howardjohn thank you for your response and let me know that from 1.16 istio will start to ignore the host.
On the other hand, I do not understand why the request respond the status code 200 if I set the host header and the port(in theory bad port, regarding the information of the config_dump).
All looks synced:

What is the best way to troubleshoot it ?

[howardjohn]

If I execute the same curl command from the istio-proxy sidecar or externally I get the status code 200.

the istio-proxy container doesn't get capture. So its not using the proxy. So presumably the service you are hitting doesn't care what header is set

[adecchi-2inno]

@howardjohn
thanks for your reply, the VS is based on host and path. My services has setup 2 VS, one for the internal traffic and one for the external traffic and both include their own gateway with mesh too.
I have not set up a domain with port 8080, but the traffic looks intercepted by istio, because when I used the host and port as header i get the response:

curl -XGET http://10.10.7.5/my-service/health -H "Host: api-internal.mycompany.com:8080"

HTTP/1.1 200 OK content-type: text/plain content-length: 77 date: Wed, 02 Nov 2022 00:10:55 GMT x-envoy-upstream-service-time: 23 appname: my-service server: envoy via: 1.1 google

Thanks

[adecchi-2inno]

@howardjohn the problem looks like it is the autority

Not Working:
':authority', 'api-internal.mycompany.com' ':path', '/my-service/health' ':method', 'GET' 'user-agent', 'curl/7.83.1' 'accept', '*/*'

Working(when send host and port header):
':authority', 'api-internal.mycompany.com:8080' ':path', '/my-service/health' ':method', 'GET' ':scheme', 'http' 'user-agent', 'curl/7.83.1' 'accept', '*/*'

If i use any random port at exception of 80 it is works.

This behavior happen inside the GKE cluster, with the internal load balancer.
No happend when hit the public load balancer inside or outside the GKE cluster or when hit the internal load balancer from outside the cluster.

[istio-policy-bot]

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2022-11-01. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.