Read Offset Checkpoint Stored in Kafka
In Kafka 0.8.2+, offset checkpoint is stored in Kafka itself, reducing the dependency on Zookeeper.
Here's how to read the offset checkpoint from the command line:
echo "exclude.internal.topics=false" > myconfigkafka-console-consumer --topic __consumer_offsets --formatter "kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter" --consumer.config myconfig --zookeeper <zookeeper> [--from-begin]
We can use kafka-console-consumer and read from the topic __consumer_offsets. There are 2 caveats.
First, we need to create a consumer configuration file with the property exclude.internal.topics set to false.
Next, we need to use the proper formatter, OffsetsMessageFormatter, conveniently :) located inside kafka.coordinator.GroupMetadataManager. Here's the code for Kafka 0.9.0 — it's also clearly commented!