http2: diagnostic use cases

[jasnell]

@nodejs/http2 @nodejs/diagnostics ... To enable better debugging and observability of http/2, we can emit http2 specific trace events. We currently can emit trace events at the C++ level and will soon be able to emit from JavaScript.

However, there is an important design decision that we need to make first and that is: what specific kinds of details should the http2 trace event log include? There are several levels of detail we can include:

• Lifecycle of Http2Session and Http2Stream objects
• Connection and request details (pseudo-header values, data flow, performance metadata)
• Timing and Performance details
• etc

Unfortunately, it's really difficult to decide what kinds of details to emit without a clear set of diagnostic use cases that we want to target so I'd like to use this thread to collect diagnostic use cases for HTTP/2 and we can go from there.

[Flarna]

In our product we capture the high level view of HTTP/2 requests (start/end + selected headers) and one of the main targets is to link them to requests on other processes. So from our point of view tracing is too low level and as the result data goes into a file (mixed with other trace categories) it's also not directly usable.
Therefore I would assume that tracing should usually target the low level details (e.g. framing, detailed timing,...). I was not involved in HTTP/2 development so I can't tell about the problems seen there, but I would assume that tracing calls to nghttp2 would help to analyze if an issue in node or in nghttp2.

But maybe adding more then one category is the key here. One for the really low level stuff to nail down any bugs/issues in the node core/nghttp2 area and one to have more end user relevant details to allow performance tuning.

[jasnell]

Closing due to lack of engagement from collaborators

[Qard]

I totally missed this issue until now...

I've added a diag-agenda tag to draw some attention to it.

[Qard]

From my perspective, lifecycle events would definitely be nice. It's a bit challenging to decide how to express http2, especially with push streams, in a coherent way.

From a lower-level perspective, it'd also be nice to have timing info for different stages of the process, like handshaking, decoding/parsing, etc. Something to provide data similar to performance.timing in the browser would be super cool.

[jasnell]

just so I'm clear on it... lifecycle events at what level? are you meaning something like Http2Session creation/completion and Http2Stream creation/completion? If so, what level of detail would be necessary? The async_hooks events in the trace event log already provide that information, but there is obviously no http2 specific detail in there (e.g. there's no detail if the Http2Stream is client or server initiated or any information about the request/response/peers/etc.

[Qard]

Yes, I was thinking Http2Session and Http2Stream. For detail, it would be nice to be able to have the necessary data to correlate push streams to their origin stream and the origin stream to the socket. It'd also help to have data to identify what the paths were for each stream and possibly headers, but that might be a bit heavy to record continuously.

[jasnell]

Heavy to record continuously, yes, but it is something that we can enable a special category for... so that it's only very selectively enabled.

[mike-kaufman]

This came up in the diag WG meeting on 2018-08-22. We don't have the depth to answer this confidently, but thought it would be useful to ping people on twitter to get input from users in the community who have specific use cases in mind. @Qard agreed to do this.