cmd/tsdbrelay: fix http: proxy error: EOF due to mixing Content-Encoding headers #1889

Merged
merged 1 commit into from Aug 26, 2016

Conversation

Projects
None yet
1 participant
@gbrayut
Contributor

gbrayut commented Aug 25, 2016

No description provided.

@gbrayut

This comment has been minimized.

Show comment
Hide comment
@gbrayut

gbrayut Aug 25, 2016

Contributor

Confirmed this fixes the periodic http: proxy error: EOF and http: proxy error: net/http: request canceled errors we were seeing. They were caused by collectors sending non-gzip requests to tsdbrelay, which then added the encoding header and forwarded to another tsdbrelay instance, causing an EOF error, which then canceled the request (see the recently added tsdbrelay.*.error metrics for more details). Those datapoints were only being saved in our primary cluster, but now they are flowing to the failover cluster as well.

I'll merge this after testing for a few more hours to make sure there are no other issues.

Contributor

gbrayut commented Aug 25, 2016

Confirmed this fixes the periodic http: proxy error: EOF and http: proxy error: net/http: request canceled errors we were seeing. They were caused by collectors sending non-gzip requests to tsdbrelay, which then added the encoding header and forwarded to another tsdbrelay instance, causing an EOF error, which then canceled the request (see the recently added tsdbrelay.*.error metrics for more details). Those datapoints were only being saved in our primary cluster, but now they are flowing to the failover cluster as well.

I'll merge this after testing for a few more hours to make sure there are no other issues.

@gbrayut gbrayut merged commit 9f7f8bf into master Aug 26, 2016

2 checks passed

bosun All checks Passed!
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@gbrayut gbrayut deleted the tsdbrelay branch Aug 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment