Add support for histogram quantiles

[srikanthccv]

Fixes https://github.com/SigNoz/engineering-pod/issues/618

This commit adds support for plotting/calculating the percentile based on the histogram buckets data.

There are two custom functions added

• histogramQuantile

histogramQuantile is enabled by adding a custom executable function in the ClickHouse using UDF. The histogramQuantile takes bucket bounds, cumulative counts and quantile as args and returns the float value. The sample usage would be like the following.

c6766f6fb6fa :) select histogramQuantile([1, 2, 4, 8, 16, 32, 64, inf], [1, 10, 12, 15, 18, 20, 25, 25], 0.9);

SELECT histogramQuantile([1, 2, 4, 8, 16, 32, 64, inf], [1, 10, 12, 15, 18, 20, 25, 25], 0.9)

Query id: c8e71e09-5f5c-4732-915a-9b2b24f9f4e1

┌─histogramQuantile([1, 2, 4, 8, 16, 32, 64, inf], [1, 10, 12, 15, 18, 20, 25, 25], 0.9)─┐
│                                                                                     48 │
└────────────────────────────────────────────────────────────────────────────────────────┘

1 rows in set. Elapsed: 0.005 sec.

The executable should be compiled for the target arch to get it to work without any issues.

[request-info]

We would appreciate it if you could provide us with more info about this issue/pr!

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

No Coverage information
0.0% Duplication

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[ankitnayan]

@srikanthccv we should check the performance comparison of the new implemented running difference vs the native

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[srikanthccv]

Here is an example of P99 for the hotrod services using this.

PromQL vs Query Builder

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[srikanthccv]

@prashant-shahi I was thinking we will maintain a separate repo signoz-udfs and use init containers to pull those changes to make them available for UDF. What do you think?

[ankitnayan]

@prashant-shahi I was thinking we will maintain a separate repo signoz-udfs and use init containers to pull those changes to make them available for UDF. What do you think?

How will this work for those who use external clickhouse clusters? The init containers should make the udfs available on hosted or external clickhouse clusters, right?

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[ankitnayan]

LGTM but more eyes can be more helpful. Feel free to merge if you are confident

[ankitnayan]

Here is an example of P99 for the hotrod services using this.

@srikanthccv I see some minor differences in the chart. Though they seem to be negligible, could you figure out any specific reason for that? We can keep a check on it

[srikanthccv]

The smoothness of the curve in the Prometheus chart varies depending on the time range [x], which is customisable and usually 5min. In our case, the runningDifference works on the subsequent points giving slightly different smoothness.

[srikanthccv]

Adding the [2m] chart, which is appropriate equivalent

[ankitnayan]

Adding the [2m] chart, which is appropriate equivalent

Almost perfect. Nice work

[github-actions]

Build Error! No Linked Issue found. Please link an issue or mention it in the body using #<issue_id>

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

No Coverage information
0.0% Duplication

[prashant-shahi]

LGTM

Looks good one level down to operations as well: