Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tier based usage metrics for historical nodes to help with autoscaling #8636

Merged
merged 2 commits into from
Oct 9, 2019

Conversation

nishantmonu51
Copy link
Member

@nishantmonu51 nishantmonu51 commented Oct 7, 2019

Add tier based usage metrics for historical nodes to help druid cluster orchestration systems understand the historical node usage and requirements.
This PR adds following metrics -

tier/required/capacity- total capacity in bytes required in each tier. Dimensions - tier
tier/total/capacity - total capacity in bytes available in a given tier. Dimension - tier
tier/historical/count - no. of historical nodes available in each tier. Dimension - tier
tier/replication/factor - configured maximum replication factor in given tier. Dimension - tier

This PR has:

  • [*] been self-reviewed.
  • [*] added documentation for new or modified features or behaviors.
  • [*] added unit tests or modified existing tests to cover new code paths.
  • [*] been tested in a test Druid cluster.

Fixes #8216

…historical autoscaling

Add tier based usage metrics for historical nodes to help druid cluster orchestration systems understand the historical node usage and requirements. Following metrics would be helpful -

tier/required/capacity- total capacity in bytes required in each tier. Dimensions - tier
tier/total/capacity - total capacity in bytes available in a given tier. Dimension - tier
tier/historical/count - no. of historical nodes available in each tier. Dimension - tier
tier/replication/factor - configured maximum replication factor in given tier. Dimension - tier
@nishantmonu51 nishantmonu51 changed the title Add tier based usage metrics for historical nodes to help with druid … Add tier based usage metrics for historical nodes to help with autoscaling Oct 7, 2019
Copy link
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems useful, lgtm 👍

@fjy fjy merged commit 0853273 into apache:master Oct 9, 2019
@jon-wei jon-wei added this to the 0.17.0 milestone Dec 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add tier based usage metrics for historical nodes to help with druid historical autoscaling
4 participants