number of unconsumed messages in a partition

[ssahukar]

Hi,

Is there a way to get the information about the number of unconsumed messages in each partition? I think the metadata information obtained from Kafka does not have this info. What does the "ISR" element in metadata structure mean?

Thanks,
Shridhar

[eapache]

ISR (and all other protocol fields) are documented in the kafka protocol spec: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-MetadataResponse

To get the number of unconsumed messages depends on how your consumer is storing offsets... you have to fetch "most recently consumed offset" from that storage location (often zookeeper), and then you can ask kafka for the most recently produced offset (Client.GetOffset). The difference is how many unconsumed messages there are.