Support 0.10.2.X versions

[Ashish123gs]

I believe we can enable the support for versions 0.10.2.X in Kafka Manager
I am using Kafka Manager for version 0.10.2.1 without any issues .

Thoughts ?

[ilavender]

Ashish123gs, were you able to configure kafka-manager consumer for SSL with 0.10.2.1 cluster?

[keithjpaulson]

I just set up a test 0.10.2.1 cluster, and all my metrics are zero; the 10.0 clusters registered with that kafka manager all show expected metrics. (I cannot say for sure it is a 10.2 issue, or if I have a config problem at this point)

Edit: My issues; I see JMX stats, except for consumers (not using ZK based offsets)

[gena01]

It seems that KIP-88 still needs to be implemented: https://cwiki.apache.org/confluence/display/KAFKA/KIP-88%3A+OffsetFetch+Protocol+Update
This allows for new API that's available in 0.10.2 via: apache/kafka#2074 which would allow direct API querying of info from broker(s).

[patelh]

@gena01 You want to get offsets from brokers using admin client instead of reading them off consumer offset topic ?

[gena01]

@patelh Yes. It would provide a more reliable way to see and monitor things. Also would reduce the necessity to scan consumer_offsets topic, which is error prone.

Wasn't sure if I should open a separate issue for this.

[patelh]

@gena01 you realize the consumer_offset topic is where the brokers read offset information ?

[gena01]

@patelh I am. And I just ran into this exact scenario. I saw the NPE that's also reported as #404. I was also troubleshooting consumer not showing up in Kafka Manager. Which could mean a broken or buggy consumer.

When I ran ./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list i saw a larger list than what's present in KM. The list included all known consumer groups even if they are not subscribing to any of the topics. Also I would trust the kafka brokers to have a better view on things than trying to scrub the back-end they are writing to.

Edit: Just saw another use case. If you run more consumers than partitions then interesting things happen, I was able to see this via kafka-consumer-groups.sh --describe. This brings a rather interesting point. I had consumers sitting and running without assigned topic/partition. Kafka-Manager just shows the subset that's actively consuming.