Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log closed connections without any response #426

Merged
merged 5 commits into from
Nov 10, 2023

Conversation

pstibrany
Copy link
Member

@pstibrany pstibrany commented Nov 6, 2023

What this PR does:

HTTP Server closes connections early (with no response, or running any handler) if client fails to send complete HTTP request headers within timeout. To help troubleshooting such attempts, this PR adds option -server.http-log-closed-connections-without-response-enabled, which enables logging of closed connections to HTTP server that received no response. First 512 bytes read from connection is also logged (while removing credentials from Authorization header).

Checklist

  • Tests updated
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Copy link
Contributor

@treid314 treid314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense, but I'm not sure how it would be used in a production environment.

Is the zero response listener something we would always enable or just turn on when we see that there's a lot of slow connections getting closed? For the second use case this makes a lot of sense since you typically don't get the user/tenant info at the gateway level and have to do backtracking via IP addresses at the LB.

@pstibrany
Copy link
Member Author

This makes sense, but I'm not sure how it would be used in a production environment.

Is the zero response listener something we would always enable or just turn on when we see that there's a lot of slow connections getting closed? For the second use case this makes a lot of sense since you typically don't get the user/tenant info at the gateway level and have to do backtracking via IP addresses at the LB.

I would propose to always enable it in environments where HTTP server is directly exposed to the internet (eg. when using AWS NLB), especially when also configuring shorter HTTP header read timeout. It could help us to make some type of currently-ignored connections more visible, and troubleshoot possible customer problems.

@pstibrany pstibrany merged commit 620b5f1 into main Nov 10, 2023
3 checks passed
@pstibrany pstibrany deleted the log-closed-connections-without-any-response branch November 10, 2023 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants