Empty data set from graphite-clickhouse triggers query retry

[matthew-hollick]

Good morning,

I hope you and your loved ones are well.

We have noticed that if graphite-clickhouse receives a query for a valid data series that does not contain any data points for the span of the query an http status 404 is returned.
This triggers the carbonapi retry mechanism which, depending on how many retries are configured, can significantly slow down the query. In my opinion this is not required as no matter how many times the (valid) query is executed the response will be the same.
The 404 response by graphite-clickhouse is by design as per go-graphite/graphite-clickhouse#39. Carbonapi is mentioned in the PR but I think the conversation ended as the user reporting the issue was specifically interested in the booking.com fork of carbonapi.

[Civil]

Good morning,

Could you please clarify what version of carbonapi you've tried? Just to confirm that this behavior still happens within latest master. As I agree, that shouldn't be the case, but as far as I remember, there were couple of changes in current master aimed to make all the "empty response" behavior more predictable.

[matthew-hollick]

Hi,

We are currently running 0.12.6 as installed from the apt repository.
If helpful I can arrange to test this behaviour with latest master.
I will get it tested with latest master

[Civil]

Actually based on that https://github.com/go-graphite/carbonapi/blob/master/zipper/helper/requests.go#L134-L138

404 response code should not trigger retry logic. Anyway, I'll try to reproduce this on my side as well with recent graphite-clickhouse version.

[Civil]

Using current master I cannot reproduce that problem. Could you please install it and check if it still happens?

[matthew-hollick]

Hi, I have tested and concur that this is no longer an issue.
It transpires that we were running quite an old version...

Many thanks.