Drop trailing dot from HOST_ADDRESS for Istio compatibility #210
Conversation
Istio on GKE currently does not support unambiguous FQDNs, (FQDN with a trailing dot). [14405](istio/istio#14405) adds this support, possibly in 1.2.x, but it is not yet available on GKE. GKE is still using 1.1 in alpha. With the trailing dot in GCP metadata server name, looks ups for GCP metadata will fail with 404. Node.js profiling agent will not work. log messages: ``` @google-cloud/profiler Failed to create profile, waiting 31.4s to try again: Error: Unexpected error determining execution environment: Unsuccessful response status code. Request failed with status code 404 @google-cloud/profiler Failed to create profile, waiting 2m 35.3s to try again: Error: Unexpected error determining execution environment: Unsuccessful response status code. Request failed with status code 404 The benchmark is running. The benchmark is running. The benchmark is running. @google-cloud/profiler Failed to create profile, waiting 4m 32.9s to try again: Error: Unexpected error determining execution environment: Unsuccessful response status code. Request failed with status code 404 The benchmark is running. ``` Istio proxy messages: ``` textPayload: "[2019-06-06T22:07:25.773Z] "GET /computeMetadata/v1/instanceHTTP/1.1" 404 NR 0 0 0 - "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" "5c8d5908-d809-445e-bd46-ad3e9cc2e9f2" "metadata.google.internal." "-" - - 169.254.169.254:80 10.28.4.11:58610" ``` This patch can be rolled back when GKE starts using Istio version that supports unambiguous FQDNs
googlebot
added
the
cla: yes
Codecov Report
@@ Coverage Diff @@
## master #210 +/- ##
=======================================
Coverage 95.65% 95.65%
=======================================
Files 1 1
Lines 46 46
Branches 10 10
=======================================
Hits 44 44
Misses 1 1
Partials 1 1
Continue to review full report at Codecov.
|
1 similar comment
Codecov Report
@@ Coverage Diff @@
## master #210 +/- ##
=======================================
Coverage 95.65% 95.65%
=======================================
Files 1 1
Lines 46 46
Branches 10 10
=======================================
Hits 44 44
Misses 1 1
Partials 1 1
Continue to review full report at Codecov.
|
|
Hi @kalyanac. Unfortunately, this isn't that simple. See #117 for more details, but the short story is that not having the FQDN leads to an excessive amount of DNS queries. We're discussing the option of using the IP address directly and never using the DNS name for the metadata service which could reduce latency and improve reliability here. If you'd like to get involved with that you can reach out to @JustinBeckwith. |
|
understood, can we change it to use the well known IP to prevent look ups? This is a blocker for Node.js profiling agent. |
|
We want to but unfortunately we need to confirm with GCE, GKE, GAE, and GCF teams first. @JustinBeckwith is working on this. |
|
Thanks for the info. Is there a tracking bug (internal or external) for that? |
Istio on GKE currently does not support unambiguous FQDNs, (FQDN with a trailing dot). 14405 adds this support, possibly in 1.2.x, but it is not yet available on GKE. GKE is still using 1.1 in alpha.
With the trailing dot in GCP metadata server name, looks ups for GCP metadata will fail with 404. Node.js profiling agent will not work.
log messages:
Istio proxy messages:
This patch can be rolled back when GKE starts using Istio version that supports unambiguous FQDNs
Fixes #<issue_number_goes_here> (it's a good idea to open an issue first for discussion)