• Skip to primary navigation
  • Skip to main content

Luc Russell

Data Engineering and Full Stack Development

  • Home
  • Blog

Kafkacat Quick Reference

08/23/2017 by Luc

Contents

  • Produce
  • Consume
  • List Topics
  • Read Messages From File
  • Generate a Test File of Messages
  • Read the Last n Messages from Partition n

kafkacat is a useful tool for working with Kafka brokers. It’s quick and very simple to use. This is a cheatsheet of some useful commands.

Produce

1
2
$ echo "foo" | kafkacat -b localhost:9092 -t mytopic
 

Consume

1
2
kafkacat -C -b localhost:9092  -t mytopic
 

List Topics

1
2
kafkacat  -b localhost:9092 -L -t mytopic
 

Read Messages From File

Read messages from a file, with each message separated by a newline:

1
2
kafkacat -P -l -b kafka:9092 -t mytopic myfile
 

Generate a Test File of Messages

Given a file containing a DATE variable, generate a set of messages with incrementing timestamp, for use with the previous example:

1
2
for i in {1..5000}; do DATE=`date -u +%d/%b/%Y:%H:%M:%S`; sed "s~DATE~$DATE~" template.msg >> myfile; done
 

Read the Last n Messages from Partition n

This example uses the -o flag to read from a specified offset. Specifying -3 allows retrieval of the last 3 messages. The -p option targets partition 0:

1
2
kafkacat -C -b kafka -t mytopic -o -3 -e -p 0
 

Share this:

  • Share
  • Click to share on LinkedIn (Opens in new window)
  • Click to share on Twitter (Opens in new window)

Filed Under: kafka Tagged With: kafka

Copyright ©2019 Luc Russell | Site by Sproutee

Copyright © 2019 · Aspire Pro on Genesis Framework · WordPress · Log in