BCP56bis: Consider expanding scope of H2/H3 stream multiplexing discussion

[LPardue]

The document makes 2 brief mentions of HTTP/2 multiplexing.

I think there is tacit knowledge about how stream multiplexing works in practice. We have experience from years of HTTP/2 deployments and interop. HTTP/3 changes things with it's removal of guaranteed ordering across streams. We also can't ignore the role that prioritization plays with multiplexing.

People wanting to use HTTP might not be with the above aspects of stream multiplexing. Through the course of discussion in #1274 it appeared to me that the designers of LL-HLS have a model for stream multiplexing usage to deliver video that may not be easily achievable with the protocol mechanics of H2 or H3, or the practical constraints that HTTP server operators apply to mitigate DoS etc. There could be emergent problems that damage their use case goals. I doubt that this is an isolated example.

So this ticket asks the question, is there benefit to the community to capture something about multiplexing in BCP56bis? Personally I think so.

[mnot]

Can you suggest some text, or at least some bullet points to seed discussion?

[LPardue]

The server push section is a pretty good model for the type of text I had in mind. Some points off the top of my head and unfiltered for if they belong in this spec or not:

• Multiplexing is a feature of the wire image and thus the connection. Don't build on assumptions that the end-to-end chain can support or preserve the properties or features of any specific HTTP version.
• Mutliplexing allows request ordering to be non-serial - don't build assumptions about request sequences in a connection
• Multiplexing allows response ordering to be non-serial - don't build assumptions about response sequences in a connection
• Multiplexing introduces per resource flow control, which impacts how applications approach reading and writing
• Multiplexing decouples message lifetime from connection lifetime - this is powerful but may also result in different types of error cases. E.g. resetting streams leading to "weird" errors compared to an error status code.
• Multiplexing has a relationship to prioritization. Don't assume there is any assured ability to influence how endpoints allocate resources to multiplexed messages
• Multiplexing can be presented in a similar way to pooling. APIs may not provide any information or control over request ordering,
  priority, etc
• Multiplexing can introduce non-obvious dependencies between messages - HoL blocking in H2, QPACK in H3
• Multiplexing credits a another form of flow control. Considerations for how multiplexed h2/h3 might get mapped to HTTP/1.1 and overwhelm an application that has previously relied on properties of HTTP/1.1 to give a flow/rate limit.
• Multiplexing may defeat connection-based controls based on assumptions of other version such as concurrent access limits, or DoS detection
• Multiplexing and connection coalescing
• Multiplexing and connection splitting (e.g. a reverse proxy that load balances or rewrites requests to different origins
• GOAWAY and the effects on multiplexing

[mnot]

We already caution against requiring a specific version of the protocol because it's a hop-by-hop protocol, and already caution against making assumptions about the relationship between a message and the connection it's on.

I'm not sure how much further we should go into the detail here. @martinthomson thoughts?

[LPardue]

I agree Mark. But OTOH the killer feature of H2 and H3 (in comparison to prior versions) is multiplexing. It seems strange to have such little text afforded to it.

[mnot]

Good point. Please write some :)

[martinthomson]

HTTP/1.1 multiplexes as well, just at a different layer. Expanding the text regarding assumptions about request-connection interactions to mention concurrency is probably a good idea. It doesn't need to be much text.

A new subsection of Section 4 might be good "Making Multiple Requests". Move the text from Section 4.10 on connection-independence to that section. Then add a probably just two things:

• Requests are made independently and you can't rely on ordering of sending within a connection to guarantee processing order. They might be sent over a multiplexed protocol by an intermediary, sent to different origin servers, or - even when pipelined on the same connection - the server can perform processing in any order (even if the responses need to be sent in the same order).

• Optimal performance of concurrent requests can depend on effective prioritization (and add a citation).

[mnot]

@LPardue @martinthomson PTAL.

[martinthomson]

I might have a suggestion for some awkward phrasing, but this is on point.

[LPardue]

The new text looks good, thanks. I made a comment about the request processing.