Binary search for large bucket count histograms #3252
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #3252 +/- ##
==========================================
- Coverage 87.10% 87.10% -0.01%
==========================================
Files 280 280
Lines 10081 10112 +31
==========================================
+ Hits 8781 8808 +27
- Misses 1300 1304 +4
|
|
This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day. |
Binary search for large bucket count histograms Update CHANGELOG.md netcoreapp3.1 was complaining ci rerun ci rerun Update MetricTestData.cs use 400 buckets
- Remove conversion to float from `FindHistogramBucketIndexBinary` - Make `DefaultHistogramCountForBinarySearch` a `const` instead of a `static readonly`
|
Closed as inactive. Feel free to reopen if this PR is still being worked on. |
|
This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day. |
|
I also had a question on how to evaluate whether it's worth the preprocessing time. There needs to be a balance. If we preprocess The issue #3248 mentioned that some customers like to have faster search, which makes sense. But is it worth to add the |
|
This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day. |
|
Closed as inactive. Feel free to reopen if this PR is still being worked on. |
| @@ -104,6 +104,46 @@ public void HistogramDistributeToAllBucketsCustom() | |||
| Assert.Equal(boundaries.Length + 1, actualCount); | |||
| } | |||
|
|
|||
| [Fact] | |||
| public void HistogramBinaryBucketTest() | |||
There was a problem hiding this comment.
I'd suggest to add e2e test for Histogram targeting the edges (default bounds, bound.length >50 etc.)
Fixes #3248
Changes
Metric.DefaultHistogramCountForBinarySearchor more buckets for a histogram, it will use binary search instead of linear search.Histogram Benchmark results from
mainwith same the same parameters.from
mic-max:hist-binary- shows slightly slower performance for small bound count and great speedup for large bound counts.For significant contributions please make sure you have completed the following items:
CHANGELOG.mdupdated for non-trivial changesAdditional Notes
To find the point at which binary search would be faster I performed only the
HistogramHotPathbenchmark with 2 similarBoundCountvalues, one above and one below the threshold value. I continued until the time taken matched for both linear and binary search which happened to be at an array size of ~50 in this scenario.