Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error handling #116

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Conversation

carrieedwards
Copy link
Collaborator

This PR creates an error package that can be reused throughout the code and will provide better detail regarding the origin of the error. This is useful for testing errors.

@grafanabot
Copy link

Go coverage report:

Click to expand.
File %
github.com/go-graphite/carbonapi/cmd/carbonapi/http/capability.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/enrichcontext.go 6.7%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/find_handlers.go 33.5%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/functions_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/helper.go 27.3%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/info_handlers.go 53.8%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/init.go 84.6%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/lbcheck_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/metrics.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/render_handler.go 50.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/tags_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/usage_handler.go 0.0%
github.com/go-graphite/carbonapi/cmd/carbonapi/http/version_handler.go 0.0%
github.com/go-graphite/carbonapi/date/date.go 78.9%
github.com/go-graphite/carbonapi/expr/consolidations/consolidations.go 45.6%
github.com/go-graphite/carbonapi/expr/expr.go 30.0%
github.com/go-graphite/carbonapi/expr/functions/absolute/function.go 80.0%
github.com/go-graphite/carbonapi/expr/functions/aggregate/function.go 82.2%
github.com/go-graphite/carbonapi/expr/functions/aggregateLine/function.go 81.0%
github.com/go-graphite/carbonapi/expr/functions/aggregateSeriesLists/function.go 73.5%
github.com/go-graphite/carbonapi/expr/functions/aggregateWithWildcards/function.go 86.7%
github.com/go-graphite/carbonapi/expr/functions/alias/function.go 83.3%
github.com/go-graphite/carbonapi/expr/functions/aliasByBase64/function.go 88.6%
github.com/go-graphite/carbonapi/expr/functions/aliasByMetric/function.go 92.9%
github.com/go-graphite/carbonapi/expr/functions/aliasByNode/function.go 84.2%
github.com/go-graphite/carbonapi/expr/functions/aliasByRedis/function.go 72.1%
github.com/go-graphite/carbonapi/expr/functions/aliasSub/function.go 77.4%
github.com/go-graphite/carbonapi/expr/functions/asPercent/function.go 77.4%
github.com/go-graphite/carbonapi/expr/functions/averageOutsidePercentile/function.go 89.3%
github.com/go-graphite/carbonapi/expr/functions/averageSeriesWithWildcards/function.go 89.4%
github.com/go-graphite/carbonapi/expr/functions/below/function.go 91.7%
github.com/go-graphite/carbonapi/expr/functions/cactiStyle/function.go 88.7%
github.com/go-graphite/carbonapi/expr/functions/changed/function.go 92.3%
github.com/go-graphite/carbonapi/expr/functions/constantLine/function.go 88.2%
github.com/go-graphite/carbonapi/expr/functions/delay/function.go 90.6%
github.com/go-graphite/carbonapi/expr/functions/derivative/function.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/divideSeries/function.go 88.9%
github.com/go-graphite/carbonapi/expr/functions/ewma/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/exclude/function.go 81.8%
github.com/go-graphite/carbonapi/expr/functions/exp/function.go 90.9%
github.com/go-graphite/carbonapi/expr/functions/exponentialMovingAverage/function.go 72.2%
github.com/go-graphite/carbonapi/expr/functions/fallbackSeries/function.go 88.2%
github.com/go-graphite/carbonapi/expr/functions/filter/function.go 84.4%
github.com/go-graphite/carbonapi/expr/functions/grep/function.go 81.8%
github.com/go-graphite/carbonapi/expr/functions/groupByNode/function.go 86.5%
github.com/go-graphite/carbonapi/expr/functions/groupByTags/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/heatMap/function.go 86.4%
github.com/go-graphite/carbonapi/expr/functions/heatMap/helpers.go 84.6%
github.com/go-graphite/carbonapi/expr/functions/highestLowest/function.go 87.1%
github.com/go-graphite/carbonapi/expr/functions/hitcount/function.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/identity/function.go 89.5%
github.com/go-graphite/carbonapi/expr/functions/integral/function.go 94.4%
github.com/go-graphite/carbonapi/expr/functions/integralByInterval/function.go 85.4%
github.com/go-graphite/carbonapi/expr/functions/integralWithReset/function.go 86.1%
github.com/go-graphite/carbonapi/expr/functions/interpolate/function.go 92.1%
github.com/go-graphite/carbonapi/expr/functions/invert/function.go 93.3%
github.com/go-graphite/carbonapi/expr/functions/isNotNull/function.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/join/function.go 89.5%
github.com/go-graphite/carbonapi/expr/functions/keepLastValue/function.go 93.2%
github.com/go-graphite/carbonapi/expr/functions/legendValue/function.go 89.1%
github.com/go-graphite/carbonapi/expr/functions/limit/function.go 82.4%
github.com/go-graphite/carbonapi/expr/functions/linearRegression/function.go 80.0%
github.com/go-graphite/carbonapi/expr/functions/logarithm/function.go 90.9%
github.com/go-graphite/carbonapi/expr/functions/logit/function.go 91.3%
github.com/go-graphite/carbonapi/expr/functions/lowPass/function.go 89.3%
github.com/go-graphite/carbonapi/expr/functions/mapSeries/function.go 88.0%
github.com/go-graphite/carbonapi/expr/functions/minMax/function.go 86.7%
github.com/go-graphite/carbonapi/expr/functions/mostDeviant/function.go 88.6%
github.com/go-graphite/carbonapi/expr/functions/moving/function.go 82.1%
github.com/go-graphite/carbonapi/expr/functions/movingMedian/function.go 86.1%
github.com/go-graphite/carbonapi/expr/functions/multiplySeriesWithWildcards/function.go 90.2%
github.com/go-graphite/carbonapi/expr/functions/nPercentile/function.go 90.0%
github.com/go-graphite/carbonapi/expr/functions/nonNegativeDerivative/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/offset/function.go 88.0%
github.com/go-graphite/carbonapi/expr/functions/offsetToZero/function.go 94.1%
github.com/go-graphite/carbonapi/expr/functions/pearson/function.go 86.1%
github.com/go-graphite/carbonapi/expr/functions/pearsonClosest/function.go 75.0%
github.com/go-graphite/carbonapi/expr/functions/perSecond/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/percentileOfSeries/function.go 78.3%
github.com/go-graphite/carbonapi/expr/functions/polyfit/function.go 87.3%
github.com/go-graphite/carbonapi/expr/functions/pow/function.go 88.9%
github.com/go-graphite/carbonapi/expr/functions/powSeries/function.go 95.0%
github.com/go-graphite/carbonapi/expr/functions/rangeOfSeries/function.go 94.4%
github.com/go-graphite/carbonapi/expr/functions/removeBelowSeries/function.go 92.3%
github.com/go-graphite/carbonapi/expr/functions/removeBetweenPercentile/function.go 89.5%
github.com/go-graphite/carbonapi/expr/functions/removeEmptySeries/function.go 90.6%
github.com/go-graphite/carbonapi/expr/functions/round/function.go 89.7%
github.com/go-graphite/carbonapi/expr/functions/scale/function.go 88.2%
github.com/go-graphite/carbonapi/expr/functions/scaleToSeconds/function.go 88.5%
github.com/go-graphite/carbonapi/expr/functions/seriesList/function.go 65.8%
github.com/go-graphite/carbonapi/expr/functions/sigmoid/function.go 91.3%
github.com/go-graphite/carbonapi/expr/functions/sinFunction/function.go 86.2%
github.com/go-graphite/carbonapi/expr/functions/slo/function.go 84.0%
github.com/go-graphite/carbonapi/expr/functions/slo/helpers.go 72.0%
github.com/go-graphite/carbonapi/expr/functions/smartSummarize/function.go 89.6%
github.com/go-graphite/carbonapi/expr/functions/sortBy/function.go 86.5%
github.com/go-graphite/carbonapi/expr/functions/sortByName/function.go 85.2%
github.com/go-graphite/carbonapi/expr/functions/squareRoot/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/substr/function.go 85.4%
github.com/go-graphite/carbonapi/expr/functions/sumSeriesWithWildcards/function.go 89.1%
github.com/go-graphite/carbonapi/expr/functions/summarize/function.go 90.5%
github.com/go-graphite/carbonapi/expr/functions/timeShift/function.go 78.0%
github.com/go-graphite/carbonapi/expr/functions/timeShiftByMetric/function.go 89.7%
github.com/go-graphite/carbonapi/expr/functions/timeShiftByMetric/misc.go 93.8%
github.com/go-graphite/carbonapi/expr/functions/transformNull/function.go 88.5%
github.com/go-graphite/carbonapi/expr/functions/tukey/function.go 80.8%
github.com/go-graphite/carbonapi/expr/functions/unique/function.go 88.9%
github.com/go-graphite/carbonapi/expr/functions/weightedAverage/function.go 85.7%
github.com/go-graphite/carbonapi/expr/helper/align.go 58.6%
github.com/go-graphite/carbonapi/expr/helper/helper.go 9.2%
github.com/go-graphite/carbonapi/expr/helper/metric/extract.go 72.7%
github.com/go-graphite/carbonapi/expr/helper/sort.go 0.0%
github.com/go-graphite/carbonapi/expr/rewrite/aboveSeries/function.go 79.3%
github.com/go-graphite/carbonapi/expr/sort.go 96.1%
github.com/go-graphite/carbonapi/expr/tags/helper.go 93.6%
github.com/go-graphite/carbonapi/expr/types/extract.go 87.5%
github.com/go-graphite/carbonapi/expr/types/list.go 0.0%
github.com/go-graphite/carbonapi/expr/types/metricheap.go 0.0%
github.com/go-graphite/carbonapi/expr/types/types.go 31.5%
github.com/go-graphite/carbonapi/expr/types/windowed.go 0.0%
github.com/go-graphite/carbonapi/pkg/parser/define.go 76.9%
github.com/go-graphite/carbonapi/pkg/parser/interface.go 0.0%
github.com/go-graphite/carbonapi/pkg/parser/internal.go 25.4%
github.com/go-graphite/carbonapi/pkg/parser/interval.go 83.8%
github.com/go-graphite/carbonapi/pkg/parser/parser.go 44.3%
github.com/go-graphite/carbonapi/util/pidfile/pidfile.go 100.0%
github.com/go-graphite/carbonapi/zipper/broadcast/broadcast_group.go 38.5%
github.com/go-graphite/carbonapi/zipper/helper/requests.go 13.3%
github.com/go-graphite/carbonapi/zipper/protocols/graphite/msgpack/type_gen.go 57.2%
github.com/go-graphite/carbonapi/zipper/protocols/irondb/irondb_group.go 2.5%
github.com/go-graphite/carbonapi/zipper/protocols/irondb/irondb_helpers.go 81.1%
github.com/go-graphite/carbonapi/zipper/protocols/prometheus/helpers/helpers.go 17.1%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/feature_set.go 19.7%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/fetch.go 0.0%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/find.go 0.0%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/tags.go 0.0%
github.com/go-graphite/carbonapi/zipper/protocols/victoriametrics/victoriametrics_group.go 7.8%
github.com/go-graphite/carbonapi/zipper/types/backend.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/errors.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/lbmethod.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/requests.go 0.0%
github.com/go-graphite/carbonapi/zipper/types/response.go 6.4%
github.com/go-graphite/carbonapi/zipper/types/stats.go 0.0%
github.com/go-graphite/carbonapi/zipper/zipper.go 0.0%
github.com/go-graphite/carbonapi/zipper/zipper_pb3.go 0.0%
total 56.1%

Go lint report:

No issues found. 😎

@npazosmendez
Copy link

npazosmendez commented Sep 16, 2022

This will be great, thank you for working on this. Two thoughts:


I wonder how cautious we should be about not breaking the public interface in terms of errors returned. The current defined errors (like ErrBadType, ErrMissingTimeseries, etc) are public so we would be breaking compatibility, but I think carbonapi is mostly used with its webserver, so the errors inside it are not really relevant for those users.


Any reason why you decided to not go with merry? I think it would make the error definitions more concise, like the ones in zipper:

var ErrResponseTypeMismatch = merry.New("type for the response doesn't match what's expected")
var ErrResponseLengthMismatch = merry.New("response length mismatch")
var ErrResponseStartTimeMismatch = merry.New("response start time mismatch")
var ErrResponseStepTimeMismatch = merry.New("response step time mismatch")
var ErrNotImplementedYet = merry.New("this feature is not implemented yet")
var ErrNotSupportedByBackend = merry.New("this feature is not supported by backend")
var ErrForbidden = merry.New("forbidden").WithHTTPCode(http.StatusForbidden)
var ErrTimeoutExceeded = merry.New("timeout while fetching Response").WithHTTPCode(http.StatusGatewayTimeout)
var ErrNonFatalErrors = merry.New("response contains non-fatal errors")
var ErrNotFound = merry.New("metric not found")
var ErrNoResponseFetched = merry.New("no responses fetched from upstream")
var ErrNoMetricsFetched = merry.New("no metrics in the Response").WithHTTPCode(http.StatusNotFound)
var ErrMaxTriesExceeded = merry.New("max tries exceeded")
var ErrFailed = merry.New("failed due to error")
var ErrFailedToFetch = merry.New("failed to fetch data from server/group")
var ErrNoRequests = merry.New("no requests to fetch")
var ErrNoTagSpecified = merry.New("no tag specified")
var ErrNoServersSpecified = merry.New("no servers specified")
var ErrConcurrencyLimitNotSet = merry.New("concurrency limit is not set")
var ErrUnmarshalFailed = merry.New("unmarshal failed")
var ErrBackendError = merry.New("error fetching data from backend").WithHTTPCode(http.StatusServiceUnavailable)
var ErrResponceError = merry.New("error while fetching Response")

Extra info could be added like this:

var BadType = merry.New("bad type")
err := BadType.Append("expected string - got int")
fmt.Println(err.Error()) // bad type: expected string - got int

I see that the errors in the parser are defined with errors

var (
// ErrMissingExpr is a parse error returned when an expression is missing.
ErrMissingExpr = errors.New("missing expression")
// ErrMissingComma is a parse error returned when an expression is missing a comma.
ErrMissingComma = errors.New("missing comma")
// ErrMissingQuote is a parse error returned when an expression is missing a quote.
ErrMissingQuote = errors.New("missing quote")
// ErrUnexpectedCharacter is a parse error returned when an expression contains an unexpected character.
ErrUnexpectedCharacter = errors.New("unexpected character")
// ErrBadType is an eval error returned when a argument has wrong type.
ErrBadType = errors.New("bad type")
// ErrMissingArgument is an eval error returned when a argument is missing.
ErrMissingArgument = errors.New("missing argument")
// ErrMissingTimeseries is an eval error returned when a time series argument is missing.
ErrMissingTimeseries = errors.New("missing time series argument")
// ErrUnknownTimeUnits is an eval error returned when a time unit is unknown to system
ErrUnknownTimeUnits = errors.New("unknown time units")
)

but it looks like that's ~5 years old, and merry was introduced later. So I think it's OK to migrate those errors to merry

@carrieedwards
Copy link
Collaborator Author

Any reason why you decided to not go with merry? I think it would make the error definitions more concise, like the ones in zipper:

I believe most of the errors I have added were were defined with fmt.Errorf() or error.New(), so they weren't originally defined as merry errors. One thing we could do is define any errors that were originally created with merry.New() as merry errors. What we could also do is merry.Wrap() with these new error types, which will make them into merry errors.

Comment on lines +15 to +18
func (e ErrMissingExpr) HTTPStatusCode() int {
return http.StatusBadRequest
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of any errors we would define here that are not bad requests, plus I don't see we are using this method anywhere yet. So I believe we can don't define these for now to simplify the code if you agree

Comment on lines +58 to +71
Exp string
Got string

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea: maybe these could be of type ExprType and the func (e ErrBadType) Error() could take care of formatting them? (i.e. EtName -> "series name", EtBool -> "bool", etc)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some cases, more than one type is acceptable, i.e. EtConst or EtString. What should we do in that case?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I can make Expected a list of type ExprType, then join them?

type ErrInvalidArgument string

func (e ErrInvalidArgument) Error() string {
return fmt.Sprintf("\"too many arguments\" %q", string(e))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably left from copypaste

@carrieedwards carrieedwards linked an issue Sep 16, 2022 that may be closed by this pull request
This commit adds the files that are specific to Grafana's fork
that are not intended to be included in the upstream. This
files are already present in the commit history (they were deleted),
but this is an attempt of a fresh start on top of the upstream's `main`.
At the moment, this should the only commit we expect to skip when
sending our changes.
carrieedwards and others added 4 commits September 16, 2022 13:01
* Fix exponentialMovingAverage panic

The panic was happening when the window size was greater than
the length of the values. Now if that happens no panics occur and
the average is returned. This matches graphiteweb.

* Add support for Graphite web toLowerCase function

* Add tests for toLowerCase function

* Update glue.go

* Update COMPATIBILITY.md to show support of toLowerCase

* Add support for Graphite web toUpperCase function

* Add tests for toUpperCase function

* Update glue.go

* Update COMPATIBILITY.md to include toUpperCase function

* Add support for Graphite web sumSeriesLists function

* Add tests for sumSeriesLists

* Update glue.go

* Update COMPATIBILITY.md to include sumSeriesLists

* Fix problems that I accidentally introduced while trying to resolve merge conflict

* Another attempt to fix previous problem

 Apparantly there is a directory that gen.go wasn't expecting and
 `go generate` in expr/functions produced `glue.go` file that didn't compile

 fix both generator and glue

* Adds Drone and Github workflow for Grafana's fork

This commit adds the files that are specific to Grafana's fork
that are not intended to be included in the upstream. This
files are already present in the commit history (they were deleted),
but this is an attempt of a fresh start on top of the upstream's `main`.
At the moment, this should the only commit we expect to skip when
sending our changes.

Co-authored-by: Nicolás Pazos <npazosmendez@gmail.com>
Co-authored-by: Vladimir Smirnov <civiloid@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve error handling
3 participants