proposal: net/http: log errors during serve()

[geofffranks]

If this should be categorized as something other than a proposal, I apologize, feel free to change as necessary. It didn't seem quite like a bug, and nothing else seemed like a good fit.

Current State

If http.Serve encounters an error while serving a request (but before sending it to any handlers via ServeHTTP), there are cases where HTTP responses are sent to clients, with no logging performed to inform the operator that a request even occurred. Notable cases include:

• unsupported transfer encoding in the http request (not response)
• unsupported/malformed HTTP protocol version
• missing/malformed host header
• invalid header name/value
• malformed method/http request
• HTTP Request Header's max length exceeded

http.Server has no hooks in place to allow for custom access log messages for these requests, but does have an ErrorLog attribute. However, it is not used to log any of these failures. It is used to log other per-request errors, such as the following:

• invalid context lengths
• superfluous/invalid attempts write calls for responses
• TLS handshake failures
• recovered panics encountered while serving a request
• if enabled, warnings regarding query params being delimited with semicolons

Proposals

Quick and Easy

We add logf() calls to the serve() function that log the failures being encountered prior to sending responses.

Thorough and Flexible

1. Add a new http.AccessLogger interface with one function: ServeHTTP(rw http.ResponseWriter, r *http.Request, next http.HandlerFunc)
2. Provide a built-in AccessLog handler for people to use if they do not wish to implement their own.
3. Add an AccessLog http.AccessLogger attribute to http.Server (defaulting to nil) that if present is called when writing any responses to the client. Currently consumers of http.Server need to provide access logging functionality on their own, but are unable to catch all responses sent. This would allow for a realitively easy drop-in + opt-in replacement for existing access log handlers.

[ianlancetaylor]

CC @neild @bradfitz

[AlexanderYastrebov]

If these are logged via ErrorLog some users may prefer to mute it, see #26918
Other users may prefer to collect metrics instead of logging so alternative option could be some kind of RequestErrorHandler hook.
Since internal errors are not exported this hook should receive status code and maybe a status text.

[neild]

http2.Server has a CountErr hook which is called when various errors are encountered:

// CountError, if non-nil, is called on HTTP/2 server errors.
// It's intended to increment a metric for monitoring, such
// as an expvar or Prometheus metric.
// The errType consists of only ASCII word characters.
CountError func(errType string)

Would a CountError on http.Server work for this case? (Called on each of these error conditions.)

[dadrus]

Hi everybody. Are there any chances, this FR is going to be addressed?