Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more size intervals to obj size histogram #18772

Merged
merged 1 commit into from Jan 13, 2024

Conversation

krisis
Copy link
Member

@krisis krisis commented Jan 11, 2024

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

New intervals:
[1024B, 64KiB)
[64KiB, 256KiB)
[256KiB, 512KiB)
[512KiB, 1MiB)

The new intervals helps us see object size distribution with higher resolution for the interval [1024B, 1MiB).

Motivation and Context

To learn more about object size distribution between 1024B to 1MiB.

How to test this PR?

 mc stat myminio/bucket --json | jq '.Usage.objectsSizesHistogram'
 {
  "BETWEEN_1024B_AND_1_MB": 1,
  "BETWEEN_1024_B_AND_64_KB": 1,
  "BETWEEN_10_MB_AND_64_MB": 0,
  "BETWEEN_128_MB_AND_512_MB": 1,
  "BETWEEN_1_MB_AND_10_MB": 0,
  "BETWEEN_256_KB_AND_512_KB": 0,
  "BETWEEN_512_KB_AND_1_MB": 0,
  "BETWEEN_64_KB_AND_256_KB": 0,
  "BETWEEN_64_MB_AND_128_MB": 0,
  "GREATER_THAN_512_MB": 0,
  "LESS_THAN_1024_B": 1
}


Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Unit tests added/updated
  • Internal documentation updated
  • Create a documentation update request here

@krisis krisis force-pushed the objsize-hist branch 2 times, most recently from 97456d7 to 6dddef5 Compare January 12, 2024 00:08
New intervals:
[1024B, 64KiB)
[64KiB, 256KiB)
[256KiB, 512KiB)
[512KiB, 1MiB)

The new intervals helps us see object size distribution with higher
resolution for the interval [1024B, 1MiB).
Copy link
Member

@vadmeste vadmeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why we are introducing a new version here ? the histogram is just a map. I am not able to see why the version change is required.

@krisis
Copy link
Member Author

krisis commented Jan 12, 2024

Any reason why we are introducing a new version here ? the histogram is just a map. I am not able to see why the version change is required.

sizeHistogram is a fixed-size array serialized as that in msgpack too. The histogram we see in prometheus is a map[string]int64 like you point out, but that's in the presentation layer.

Copy link
Member

@vadmeste vadmeste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@harshavardhana harshavardhana merged commit cba3dd2 into minio:master Jan 13, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants