Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental Mixer-less telemetry lacks parity with Mixer #17231

Closed
jshaughn opened this issue Sep 19, 2019 · 6 comments
Assignees
Milestone

Comments

@jshaughn
Copy link

@jshaughn jshaughn commented Sep 19, 2019

I realize that mixer-less telemetry is WIP/experimental in 1.3.0 but wanted to officially be able to track issues in the new telemetry that affect Kiali or potentially other consumers.

After configuring mixer-less using these instructions I found the following differences in istio_requests_total (may not be an exhaustive list), using bookinfo, that negatively affect Kiali:

With Mixer:

destination_service="ratings.bookinfo.svc.cluster.local"
destination_service_name="ratings"

Mixer-less:

destination_service="ratings:9080"     // maybe incorrectly set to request.host?
destination_service_name="ratings-v1"  // maybe incorrectly set to destination.workload.name?

I was expecting parity in values with or without mixer.

[X ] Policies and Telemetry

> istioctl version --remote
client version: 1.3.0
control plane version: 1.3.0

> kubectl version
oc v3.11.0+maistra-0.11.0+f2e7517
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://x.x.x.x:8443
kubernetes v1.11.0+d4cacc0

Istio installed via helm charts on a local OpenShift 3.11 on an F28 VM.

@douglas-reid

This comment has been minimized.

Copy link
Contributor

@douglas-reid douglas-reid commented Sep 20, 2019

@jshaughn I've noticed this as well. Thanks for filing, and catching this early.

@howardjohn

This comment has been minimized.

Copy link
Member

@howardjohn howardjohn commented Sep 27, 2019

This is marked as a 1.4 P0, do we have an owner?

@mandarjog

This comment has been minimized.

Copy link
Contributor

@mandarjog mandarjog commented Oct 13, 2019

Here is an example of v2 telemetry full metric

--- istio_req.json	2019-10-11 11:30:32.000000000 -0700
+++ envoy_req.json	2019-10-11 11:30:21.000000000 -0700
@@ -1,30 +1,26 @@
-app="svc05-0"
 connection_security_policy="none"
-date="1570471790"
 destination_app="svc05-0"
 destination_principal="unknown"
-destination_service="svc05-0.local"
-destination_service_name="svc05-0v1"
+destination_service="svc05-0.service-graph05.svc.cluster.local"
+destination_service_name="svc05-0"
 destination_service_namespace="service-graph05"
 destination_version="v1"
 destination_workload="svc05-0v1"
 destination_workload_namespace="service-graph05"
-endpoint="15090"
-instance="10.36.13.211:15090"
-job="svc05-0"
-namespace="service-graph05"
-pod="svc05-0v1-69b8c58b55-kghnx"
-pod_name="svc05-0v1-69b8c58b55-kghnx"
-pod_template_hash="69b8c58b55"
+endpoint="prometheus"
+instance="10.36.10.228:42422"
+job="istio-telemetry"
+namespace="istio-system"
+permissive_response_code="none"
+permissive_response_policyid="none"
+pod="istio-telemetry-7b6c7996c7-5fm65"
 reporter="destination"
 request_protocol="http"
 response_code="200"
 response_flags="-"
-role="service"
-service="svc05-0"
-source_app="unknown"
+service="istio-telemetry"
+source_app="istio-ingressgateway"
 source_principal="unknown"
 source_version="unknown"
 source_workload="istio-ingressgateway"
 source_workload_namespace="istio-system"
-version="v1"
@bianpengyuan

This comment has been minimized.

Copy link
Contributor

@bianpengyuan bianpengyuan commented Oct 13, 2019

The meaningful diffs are:

  1. mixerless fills destination_service with host header and destination_service_name with workload name, while mixer fills in service fqdn and short name.
  2. mixerless does hot fill permissive_response_code and permissive_response_policyid which are from authn filter.
  3. mixerless does not fill in app label.
@mandarjog

This comment has been minimized.

Copy link
Contributor

@mandarjog mandarjog commented Oct 13, 2019

Yes. Need to check if app label missing is specific to source ingress or also sidecars.
Need to see what app label is used by ingress.

@bianpengyuan

This comment has been minimized.

Copy link
Contributor

@bianpengyuan bianpengyuan commented Oct 14, 2019

Actually I miss read the diff a bit. app and version labels are missing in mixer metrics, not mixerless metrics. Looks like prometheus adds those implicitly during scraping, so we just need to fix the first two diffs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
5 participants
You can’t perform that action at this time.