GRPC Proxyless Support

[anuragagarwal561994]

When enabling istio on proxyless grpc the edges are missing.

I understand that unless the metrics from the sidecar are available it will be not easy for kiali to understand what is happening.

Can a workaround be thought of for this use case if any exists?

[anuragagarwal561994]

@jshaughn thanks for starting a discussion on it.

I was thinking if I can use Kiali extension API support to provide with a POC and then from there we can see if this can be included in this repo.

Since I am new to Kiali for contribution, I would like to understand what needs to be thought upon for this or any guidance that would be helpful for me to start this?

[anuragagarwal561994]

So there are traffic telemetry that grpc Proxyless produces but there is problem with standardisation.

For example, I could find 3 vendors:

1. OpenTelemetry
2. OpenCensus (as suggested on grpc doc)
3. Micrometer (java specific)

From the above and more vendor agnostic solution I could find OpenCensus to dona good job in producing the following:

1. Response / Request size
2. Request / Response counts
3. Request / Response latency
4. Error codes

There can be more but I guess these seems reasonable enough.

I will need to check once how the endpoints are recorded in this like if we want to link it with vs or ds.

If endpoints are properly defined I guess other things can be fetched from istio configurations for linking.

And the above metrics should more or less cover a decent chunk.

If needed an extension to opencensus can also be explored which translates it to a format that kiali can understand so that kiali is not vendor bound.

[nrfox]

@anuragagarwal561994 does proxyless grpc have a /metrics endpoint that already exports the istio standard metrics? If so, you may need to add the prometheus scrape annotations manually to your pod to start getting prom to scrape those endpoints. Once your metrics are in prometheus, Kiali should be able to show traffic details. Kiali doesn't support other datastores for metrics other than prometheus.

[anuragagarwal561994]

There is a doc in kiali documentation for which metrics it consumes, if I use that and kind of add a conversion for starters from grpc to istio standard metrics I guess that should work.

I will give a hands on that.

[jshaughn]

Maybe you should start a topic in Istio Slack's telemetry channel and ask about plans they may have for offering standard Istio metric telemetry for proxyless-GRPC. I wouldn't want to see you have to create some sort of bridge if they have a plan to generate information natively.

[anuragagarwal561994]

@jshaughn can we customise the metric used to build edges and traffic rate.

I was going through the documentation and identified that if traces are present then edges can be built from that.

I can also see we can make custom dashboards as well. Just wanted to understand if we can customise the traffic rate part based on another metric.

[jmazzitelli]

can we customise the metric used to build edges and traffic rate

No, those are fixed and come from Istio telemetry. You cannot tell Kiali to use different metrics.

[anuragagarwal561994]

Can we represent these as TCP metrics like for kafka. I have been going through issues and articles but not able to clearly understand how everything ties up to show kafka event messages and message rate on kiali dashboard.

[jmazzitelli]

You won't see event messages data because to Istio those are just TCP byte traffic and they log it as such (which is why Kiali shows it as TCP traffic). Here's an article written by a previous Kiali team member that talks about how Kiali can be used with Kafka or other messaging systems like that - it was a three-part series but the third part talks about Kafka; this might be helpful:

• https://medium.com/kialiproject/trace-my-mesh-part-3-3-e1e64795e61f