Ambient Mesh - Validate Telemetry

[josunect]

At the moment, there are a couple of issues:

• It is not possible to see service nodes. Looking at the telemetry data reported by ztunnel, the destination_service is unknown: sum(istio_tcp_sent_bytes_total{app="ztunnel"}) by (reporter, destination_canonical_service, destination_service, destination_service_name)

{destination_canonical_service="details", destination_service="unknown", destination_service_name="unknown", reporter="destination"}
{destination_canonical_service="details", destination_service="unknown", destination_service_name="unknown", reporter="source"}
{destination_canonical_service="istio-ingressgateway", destination_service="unknown", destination_service_name="unknown", reporter="source"}
{destination_canonical_service="productpage", destination_service="unknown", destination_service_name="unknown", reporter="destination"

• When we create a waypoint proxy, some edged are duplicated with http and tcp, because ztunnel reports everything as tcp traffic.

L4 telemetry:

With waypoint proxy:

Epic subtask: #5476

istio/istio#46169

[josunect]

• In minikube, the telemetry looks like (Tested in Istio 1.18):

(Perhaps there is no processing in L4 for the local node?)

[josunect]

After test this PR: istio/istio#46169

It is possible to see the service nodes. But, now there are some duplicated edges.

sum(istio_tcp_sent_bytes_total{app="ztunnel"}) by (reporter, destination_canonical_service, destination_service, destination_service_name)

{destination_canonical_service="details", destination_service="details.bookinfo.svc.cluster.local", destination_service_name="details", reporter="source"} | 176852
{destination_canonical_service="details", destination_service="unknown", destination_service_name="unknown", reporter="destination"} | 112632
{destination_canonical_service="productpage", destination_service="productpage.bookinfo.svc.cluster.local", destination_service_name="productpage", reporter="source"} | 2542904
{destination_canonical_service="productpage", destination_service="unknown", destination_service_name="unknown", reporter="destination"} | 2529671
{destination_canonical_service="ratings", destination_service="ratings.bookinfo.svc.cluster.local", destination_service_name="ratings", reporter="source"} | 69264
{destination_canonical_service="ratings", destination_service="unknown", destination_service_name="unknown", reporter="destination"} | 73668
{destination_canonical_service="reviews", destination_service="reviews.bookinfo.svc.cluster.local", destination_service_name="reviews", reporter="source"} | 282662
{destination_canonical_service="reviews", destination_service="unknown", destination_service_name="unknown", reporter="destination"}

I was debugging the edges generation and it looks like the extra edge is generated with the query:

sum(rate(istio_tcp_received_bytes_total{reporter="destination",destination_workload_namespace="ambient"} [15306s])) by (source_cluster,source_workload_namespace,source_workload,source_canonical_service,source_canonical_revision,destination_cluster,destination_service_namespace,destination_service,destination_service_name,destination_workload_namespace,destination_workload,destination_canonical_service,destination_canonical_revision,response_flags) > 0

{destination_canonical_revision="latest", destination_canonical_service="tcpserver", destination_cluster="Kubernetes", destination_service="unknown", destination_service_name="unknown", destination_service_namespace="unknown", destination_workload="tcpserver", destination_workload_namespace="ambient", response_flags="-", source_canonical_revision="latest", source_canonical_service="debugbox", source_cluster="Kubernetes", source_workload="debugbox", source_workload_namespace="ambient"}

In the query destination telemetry: https://github.com/kiali/kiali/blob/master/graph/telemetry/istio/istio.go#L235

There are 3 edges from the debugbox node:

edges": [
      {
        "data": {
          "id": "1a54005a7c18bf3d28a5def62bc38256",
          "source": "0ef8a87188caa5c6a3e637c764420883",
          "target": "17ce12e7f26ee796d0538bea255c0e90",
          "traffic": {
            "protocol": "tcp",
            "rates": {
              "tcp": "0.01"
            },
            "responses": {
              "-": {
                "flags": {
                  "-": "100.0"
                },
                "hosts": {
                  "tcpserver.ambient.svc.cluster.local": "100.0"
                }
              }
            }
          }
        }
      },
      {
        "data": {
          "id": "522c03d0469c858928b947084b2cfb75",
          "source": "0ef8a87188caa5c6a3e637c764420883",
          "target": "24b3a0bfcc52c7df84de1cc97ca81f42",
          "traffic": {
            "protocol": "tcp",
            "rates": {
              "tcp": "0.007"
            },
            "responses": {
              "-": {
                "flags": {
                  "-": "100.0"
                },
                "hosts": {
                  "httpserver.ambient.svc.cluster.local": "100.0"
                }
              }
            }
          }
        }
      },
      {
        "data": {
          "id": "7344506aac9e77db6bac915046a8a693",
          "source": "0ef8a87188caa5c6a3e637c764420883",
          "target": "2f19c182bef63e07d0c334bc11a6f84b",
          "traffic": {
            "protocol": "tcp",
            "rates": {
              "tcp": "0.02"
            },
            "responses": {
              "-": {
                "flags": {
                  "-": "100.0"
                },
                "hosts": {
                  "unknown": "100.0"
                }
              }
            }
          }
        }
      },

[josunect]

Validated:

[josunect]

Tested today (With the updated Istio image) and the queries seems to have unknown data again:

[josunect]

Working as expected in Istio 1.20