Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pulsar-storm] add more metrics to troubleshoot spout throughput #4280

Merged
merged 1 commit into from
May 15, 2019

Conversation

rdhabalia
Copy link
Contributor

Motivation

Many time user sees lower throughput in pulsar-spout even though standalone consumer can consume such msgRate easily. It would be hard to debug user's topology without enough information so, adding two metrics which can impact spout throughput.

  • number of message filed: spout sleeps when it sees failed message so, it's important to have visibility of that count
  • number of times spout-thread not found the message in queue: spout topology internally sleeps if it doesn't see any emitted tuple in collector after triggering nextTuple() api.

This metrics gives more visibility about consumer throughput.

@rdhabalia rdhabalia added this to the 2.4.0 milestone May 15, 2019
@rdhabalia rdhabalia self-assigned this May 15, 2019
Copy link
Contributor

@merlimat merlimat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@merlimat
Copy link
Contributor

run integration tests
run java8 tests

@rdhabalia
Copy link
Contributor Author

rerun java8 tests

Copy link
Member

@jiazhai jiazhai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@jiazhai jiazhai merged commit 2c09ff2 into apache:master May 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants