Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check validity of consolidation function in sortBy #102

Merged
merged 3 commits into from
Sep 8, 2022

Conversation

carrieedwards
Copy link
Collaborator

This PR fixes a bug found in the sortBy() function, in which the specified consolidation function was not checked to verify that it was a valid function. This would cause a panic to occur, because in consolidations.SummarizeValues, the default case in the switch statement involves a string split which assumes that the consolidation function is 'p50'-'p999'. If the specified consolidation function was not 'p50'-'p999', then the panic would occur on the string split.

@grafanabot

This comment has been minimized.

@carrieedwards carrieedwards marked this pull request as ready for review September 8, 2022 14:46
@ywwg
Copy link

ywwg commented Sep 8, 2022

can you add a regression test for this case?

@carrieedwards carrieedwards force-pushed the cedwards/fix-consolidation-summarizer branch from b6ab967 to 2315796 Compare September 8, 2022 15:05
@grafanabot

This comment has been minimized.

@grafanabot

This comment has been minimized.

@grafanabot

This comment has been minimized.

@@ -112,6 +112,9 @@ func (f *smartSummarize) Do(ctx context.Context, e parser.Expr, from, until int6
}

if t >= bucketEnd {
if err := consolidations.CheckValidConsolidationFunc(summarizeFunction); err != nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better if we do this only once, and just after the summarizeFunction is obtained, in line 53

@@ -124,6 +127,9 @@ func (f *smartSummarize) Do(ctx context.Context, e parser.Expr, from, until int6

// last partial bucket
if bucketItems > 0 {
if err := consolidations.CheckValidConsolidationFunc(summarizeFunction); err != nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

expr/consolidations/consolidations.go Show resolved Hide resolved
@@ -59,6 +59,9 @@ func (f *legendValue) Do(ctx context.Context, e parser.Expr, from, until int64,
nameBuf.Grow(len(r.Name) + len(methods)*5)
nameBuf.WriteString(r.Name)
for _, method := range methods {
if err := consolidations.CheckValidConsolidationFunc(method); err != nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we could move this check to line 51 so it's done at the beginning and some of the allocations are avoided

@@ -157,6 +160,9 @@ func (f *summarize) Do(ctx context.Context, e parser.Expr, from, until int64, va

// last partial bucket
if bucketItems > 0 {
if err := consolidations.CheckValidConsolidationFunc(summarizeFunction); err != nil {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as for the smartSummarize, we can move these checks just after the summarizeFunction is obtained from the arguments

@grafanabot

This comment has been minimized.

@carrieedwards carrieedwards force-pushed the cedwards/fix-consolidation-summarizer branch from b89ca07 to a4053e5 Compare September 8, 2022 17:11
@grafanabot

This comment has been minimized.

Comment on lines 52 to 54
if err != nil {
return ErrInvalidConsolidationFunc.WithMessage("invalid consolidation " + functionName)
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be deleted.
Also nitnit: the else branch is not necessary

expr/consolidations/consolidations.go Show resolved Hide resolved
@carrieedwards carrieedwards force-pushed the cedwards/fix-consolidation-summarizer branch 2 times, most recently from 5db03b4 to 6e4ee3b Compare September 8, 2022 17:25
@grafanabot

This comment has been minimized.

return nil
}

return ErrInvalidConsolidationFunc.WithMessage("invalid consolidation " + functionName)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I meant the whole return statement isn't needed, since it would just return at line 55

@carrieedwards carrieedwards force-pushed the cedwards/fix-consolidation-summarizer branch from 6e4ee3b to d34d68e Compare September 8, 2022 17:56
@grafanabot
Copy link

Go coverage report:

Click to expand.
File %
github.com/go-graphite/carbonapi/cmd/carbonapi/http/capability.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/enrichcontext.go 6.7%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/find_handlers.go 33.5%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/functions_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/helper.go 27.3%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/info_handlers.go 53.8%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/init.go 84.6%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/lbcheck_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/metrics.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/render_handler.go 50.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/tags_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/usage_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/version_handler.go 0.0%
github.com/go-graphite/carbonapi/date/date.go 78.9%
github.com/go-graphite/carbonapi/expr/consolidations/consolidations.go 45.4%
github.com/go-graphite/carbonapi/expr/expr.go 30.0%
github.com/go-graphite/carbonapi/expr/functions/absolute/function.go 80.0%
github.com/go-graphite/carbonapi/expr/functions/aggregate/function.go 82.2%
github.com/go-graphite/carbonapi/expr/functions/aggregateLine/function.go 81.0%
github.com/go-graphite/carbonapi/expr/functions/aggregateSeriesLists/function.go 73.5%
github.com/go-graphite/carbonapi/expr/functions/aggregateWithWildcards/function.go 86.7%
github.com/go-graphite/carbonapi/expr/functions/alias/function.go 83.3%
github.com/go-graphite/carbonapi/expr/functions/aliasByBase64/function.go 88.9%
github.com/go-graphite/carbonapi/expr/functions/aliasByMetric/function.go 92.9%
github.com/go-graphite/carbonapi/expr/functions/aliasByNode/function.go 83.3%
github.com/go-graphite/carbonapi/expr/functions/aliasByRedis/function.go 72.1%
github.com/go-graphite/carbonapi/expr/functions/aliasSub/function.go 77.4%
github.com/go-graphite/carbonapi/expr/functions/asPercent/function.go 77.4%
github.com/go-graphite/carbonapi/expr/functions/averageOutsidePercentile/function.go 89.3%
github.com/go-graphite/carbonapi/expr/functions/averageSeriesWithWildcards/function.go 89.4%
github.com/go-graphite/carbonapi/expr/functions/below/function.go 91.7%
github.com/go-graphite/carbonapi/expr/functions/cactiStyle/function.go 88.7%
github.com/go-graphite/carbonapi/expr/functions/changed/function.go 92.3%
github.com/go-graphite/carbonapi/expr/functions/constantLine/function.go 88.2%
github.com/go-graphite/carbonapi/expr/functions/delay/function.go 90.6%
github.com/go-graphite/carbonapi/expr/functions/derivative/function.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/divideSeries/function.go 88.1%
github.com/go-graphite/carbonapi/expr/functions/ewma/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/exclude/function.go 81.8%
github.com/go-graphite/carbonapi/expr/functions/exp/function.go 90.9%
github.com/go-graphite/carbonapi/expr/functions/exponentialMovingAverage/function.go 72.2%
github.com/go-graphite/carbonapi/expr/functions/fallbackSeries/function.go 86.7%
github.com/go-graphite/carbonapi/expr/functions/filter/function.go 84.4%
github.com/go-graphite/carbonapi/expr/functions/grep/function.go 81.8%
github.com/go-graphite/carbonapi/expr/functions/groupByNode/function.go 86.5%
github.com/go-graphite/carbonapi/expr/functions/groupByTags/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/heatMap/function.go 86.4%
github.com/go-graphite/carbonapi/expr/functions/heatMap/helpers.go 84.6%
github.com/go-graphite/carbonapi/expr/functions/highestLowest/function.go 87.1%
github.com/go-graphite/carbonapi/expr/functions/hitcount/function.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/identity/function.go 89.5%
github.com/go-graphite/carbonapi/expr/functions/integral/function.go 94.4%
github.com/go-graphite/carbonapi/expr/functions/integralByInterval/function.go 85.4%
github.com/go-graphite/carbonapi/expr/functions/integralWithReset/function.go 86.1%
github.com/go-graphite/carbonapi/expr/functions/interpolate/function.go 92.1%
github.com/go-graphite/carbonapi/expr/functions/invert/function.go 93.3%
github.com/go-graphite/carbonapi/expr/functions/isNotNull/function.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/join/function.go 89.5%
github.com/go-graphite/carbonapi/expr/functions/keepLastValue/function.go 93.2%
github.com/go-graphite/carbonapi/expr/functions/legendValue/function.go 89.1%
github.com/go-graphite/carbonapi/expr/functions/limit/function.go 82.4%
github.com/go-graphite/carbonapi/expr/functions/linearRegression/function.go 80.0%
github.com/go-graphite/carbonapi/expr/functions/logarithm/function.go 90.9%
github.com/go-graphite/carbonapi/expr/functions/logit/function.go 91.3%
github.com/go-graphite/carbonapi/expr/functions/lowPass/function.go 89.3%
github.com/go-graphite/carbonapi/expr/functions/mapSeries/function.go 88.0%
github.com/go-graphite/carbonapi/expr/functions/minMax/function.go 86.7%
github.com/go-graphite/carbonapi/expr/functions/mostDeviant/function.go 88.6%
github.com/go-graphite/carbonapi/expr/functions/moving/function.go 82.1%
github.com/go-graphite/carbonapi/expr/functions/movingMedian/function.go 86.1%
github.com/go-graphite/carbonapi/expr/functions/multiplySeriesWithWildcards/function.go 90.2%
github.com/go-graphite/carbonapi/expr/functions/nPercentile/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/nonNegativeDerivative/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/offset/function.go 88.0%
github.com/go-graphite/carbonapi/expr/functions/offsetToZero/function.go 94.1%
github.com/go-graphite/carbonapi/expr/functions/pearson/function.go 86.1%
github.com/go-graphite/carbonapi/expr/functions/pearsonClosest/function.go 75.0%
github.com/go-graphite/carbonapi/expr/functions/perSecond/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/percentileOfSeries/function.go 78.3%
github.com/go-graphite/carbonapi/expr/functions/polyfit/function.go 87.3%
github.com/go-graphite/carbonapi/expr/functions/pow/function.go 88.9%
github.com/go-graphite/carbonapi/expr/functions/powSeries/function.go 95.0%
github.com/go-graphite/carbonapi/expr/functions/rangeOfSeries/function.go 94.3%
github.com/go-graphite/carbonapi/expr/functions/removeBelowSeries/function.go 92.3%
github.com/go-graphite/carbonapi/expr/functions/removeBetweenPercentile/function.go 89.2%
github.com/go-graphite/carbonapi/expr/functions/removeEmptySeries/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/round/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/scale/function.go 88.6%
github.com/go-graphite/carbonapi/expr/functions/scaleToSeconds/function.go 88.5%
github.com/go-graphite/carbonapi/expr/functions/seriesList/function.go 67.2%
github.com/go-graphite/carbonapi/expr/functions/sigmoid/function.go 91.3%
github.com/go-graphite/carbonapi/expr/functions/sinFunction/function.go 86.2%
github.com/go-graphite/carbonapi/expr/functions/slo/function.go 84.0%
github.com/go-graphite/carbonapi/expr/functions/slo/helpers.go 72.0%
github.com/go-graphite/carbonapi/expr/functions/smartSummarize/function.go 89.6%
github.com/go-graphite/carbonapi/expr/functions/sortBy/function.go 86.5%
github.com/go-graphite/carbonapi/expr/functions/sortByName/function.go 85.2%
github.com/go-graphite/carbonapi/expr/functions/squareRoot/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/substr/function.go 85.4%
github.com/go-graphite/carbonapi/expr/functions/sumSeriesWithWildcards/function.go 89.1%
github.com/go-graphite/carbonapi/expr/functions/summarize/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/timeShift/function.go 78.0%
github.com/go-graphite/carbonapi/expr/functions/timeShiftByMetric/function.go 89.7%
github.com/go-graphite/carbonapi/expr/functions/timeShiftByMetric/misc.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/transformNull/function.go 88.5%
github.com/go-graphite/carbonapi/expr/functions/tukey/function.go 80.8%
github.com/go-graphite/carbonapi/expr/functions/unique/function.go 88.9%
github.com/go-graphite/carbonapi/expr/functions/weightedAverage/function.go 85.7%
github.com/go-graphite/carbonapi/expr/helper/align.go 60.7%
github.com/go-graphite/carbonapi/expr/helper/helper.go 9.2%
github.com/go-graphite/carbonapi/expr/helper/metric/extract.go 72.7%
github.com/go-graphite/carbonapi/expr/helper/sort.go 0.0%
github.com/go-graphite/carbonapi/expr/rewrite/aboveSeries/function.go 79.3%
github.com/go-graphite/carbonapi/expr/sort.go 96.1%
github.com/go-graphite/carbonapi/expr/tags/helper.go 93.6%
github.com/go-graphite/carbonapi/expr/types/extract.go 84.6%
github.com/go-graphite/carbonapi/expr/types/list.go 0.0%
github.com/go-graphite/carbonapi/expr/types/metricheap.go 0.0%
github.com/go-graphite/carbonapi/expr/types/types.go 31.5%
github.com/go-graphite/carbonapi/expr/types/windowed.go 0.0%
github.com/go-graphite/carbonapi/pkg/parser/define.go 76.9%
github.com/go-graphite/carbonapi/pkg/parser/interface.go 0.0%
github.com/go-graphite/carbonapi/pkg/parser/internal.go 9.4%
github.com/go-graphite/carbonapi/pkg/parser/interval.go 83.8%
github.com/go-graphite/carbonapi/pkg/parser/parser.go 44.3%
github.com/go-graphite/carbonapi/util/pidfile/pidfile.go 100.0%
github.com/go-graphite/carbonapi/zipper/broadcast/broadcast_group.go 38.5%
github.com/go-graphite/carbonapi/zipper/helper/requests.go 13.3%
github.com/go-graphite/carbonapi/zipper/protocols/graphite/msgpack/type_gen.go 57.2%
github.com/go-graphite/carbonapi/zipper/protocols/irondb/irondb_group.go 2.5%
github.com/go-graphite/carbonapi/zipper/protocols/irondb/irondb_helpers.go 81.1%
github.com/go-graphite/carbonapi/zipper/protocols/prometheus/helpers/helpers.go 17.1%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/feature_set.go 19.7%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/fetch.go 0.0%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/find.go 0.0%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/tags.go 0.0%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/victoriametrics_group.go 7.8%
github.com/go-graphite/carbonapi/zipper/types/backend.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/errors.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/lbmethod.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/requests.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/response.go 6.4%
github.com/go-graphite/carbonapi/zipper/types/stats.go 0.0%
github.com/go-graphite/carbonapi/zipper/zipper.go 0.0%
github.com/go-graphite/carbonapi/zipper/zipper_pb3.go 0.0%
total 56.0%

Go lint report:

No issues found. 😎

@carrieedwards carrieedwards merged commit 8098192 into main Sep 8, 2022
@carrieedwards carrieedwards linked an issue Sep 9, 2022 that may be closed by this pull request
carrieedwards added a commit that referenced this pull request Sep 14, 2022
…izer (#102)

* Check validity of consolidation function in sortBy

* Add test for checking validity of consolidation functions

* Update validity checker function and add tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SortBy function panics if invalid consolidation function passed in
4 participants