Support :

Kafka Consumer groups and Lag

Published by Murali Basani on

When kafka consumers are subscribing to a topic, as and when messages are produced, they are consumed by the consumers. But due to high traffic or latency or high load issues, there would be a lag in consuming the messages.

With Kafka binaries it is possible to know the lag of every consumer group, and per topic even.

Display all consumer groups

/bin/kafka-consumer-groups --bootstrap-server localhost:9092  --list

Describe a consumer group

/bin/kafka-consumer-groups --bootstrap-server localhost:9092 --group consumergroup  --describe

This will display all the connected consumer clients, subscribing to each topic, and the current offset and lag. Sample output below.

GROUP           TOPIC   PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                                            HOST            CLIENT-ID
consumergroup testtopic 0 376 376 0 consumer-consumergroup-1-e2e668bd-a2cf-4dca-b27c-68f6d89f6859 / consumer-consumergroup-1
consumergroup testtopic 1 364 374 10 consumer-consumergroup-1-e2e668bd-a2cf-4dca-b27c-68f6d89f6859 / consumer-consumergroup-2

Describe a consumer group with SSL

/bin/kafka-consumer-groups --bootstrap-server localhost:9093 --group consumergroup --describe --command-config


Leave a Reply

Your email address will not be published. Required fields are marked *