Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a simple topic consumer tool #439

Merged
merged 1 commit into from
May 1, 2015
Merged

Add a simple topic consumer tool #439

merged 1 commit into from
May 1, 2015

Conversation

wvanbergen
Copy link
Contributor

It will consume all partitions for a given topic and write the messages to STDOUT.

@Shopify/kafka

eapache
eapache reviewed Apr 30, 2015
View reviewed changes
tools/kafka-console-topicconsumer/kafka-console-topicconsumer.go

wg.Add(1)
go func(partition int32, pc sarama.PartitionConsumer) {
defer wg.Done()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't have to be part of the wait group, it must terminate before the messages goroutine does (except in the case of OffsetOutOfRange when I don't think it matters)

@eapache
Copy link
Contributor

eapache commented Apr 30, 2015

Little things, but LGTM

@bouk
Copy link

bouk commented Apr 30, 2015

Maybe "kafka-console-consumer" is a better name, not like there's other things to consume in Kafka 😃

@eapache
Copy link
Contributor

eapache commented Apr 30, 2015

We actually already have a kafka-console-partitionconsumer which only consumes a single partition, but I wouldn't mind just merging them into one kafka-console-consumer :)

@wvanbergen
Copy link
Contributor Author

I think kafka-console-consumer should be built on top the high-level consumer once we have it.

@wvanbergen
Add a simple topic consumer tool, which will consume all partitions f…
12f24c3 
…or a given topic and write the messages to STDOUT.
wvanbergen added a commit that referenced this pull request May 1, 2015
@wvanbergen
Merge pull request #439 from Shopify/topic_consumer_tool
b5ea195 
Add a simple topic consumer tool
@wvanbergen wvanbergen merged commit b5ea195 into master May 1, 2015
@wvanbergen wvanbergen deleted the topic_consumer_tool branch May 1, 2015 00:06
@wvanbergen
Copy link
Contributor Author

@Shopify/kafka Feel free to still comment on this further; there's definitely ways to improve this tool, but I had an urgent need for it, so I merge this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@wvanbergen @eapache @bouk