New push API

[kazu-yamamoto]

Random thoughts based on discussions in the Japanese HTTP/2 community:

How does a server know contents to be pushed?

• Learning based on Referer: like Jetty.
• Push manifest: https://github.com/GoogleChrome/http2-push-manifest/

How does a server know already-cached contents by a client?

• Cache digest: https://tools.ietf.org/html/draft-kazuho-h2-cache-digest

What about API?

• New APIs are necessary. To maintain backward compatibility, we should define HTTP2Application as @awpr did.

[kazu-yamamoto]

Story:

• An HTTP2Appication (or HTTP2Middleware) learns contents to be pushed somehow.
• A client sends a GET request and a cache digest.
• Warp stores the cache digest to Stream and launches the HTTP2Application with Request.
• The HTTP2Application returns Response with additional push information [(ByteString,Response)] where ByteString indicates path information.
• Warp filters push information based on the cache digest.
• Warp creates PushPromises based on path information and enqueues them as control frames to ensure that they will be sent before Responses to be pushed.
• Warps enqueues Responses to be pushed with higher priority than the actual Response for GET.

[awpr]

Some thoughts:

• I originally had push exposed via an IO action to enqueue a promise, to match the semantics of the protocol more closely: server pushes can be initiated by the server at any time, they needn't be at the beginning of a stream. I don't know of any use cases that require this extra flexibility, so I don't feel strongly about it; just recording the motivation.
• I originally had promise frames enqueued in-band with the data of the associated stream. This trivially guarantees ordering w.r.t. the contents of the stream, which is required by the spec (the push must be sent before any data referencing its resource). I haven't looked closely enough to know whether sending it as a control frame would give the same guarantee.
• On how to discover pushed resources: it seems that Yesod's document-generating monad could be extended easily enough with a list of resources to push -- then widgets like image tags or stylesheets would add their referents to the list. This would be more precise than guessing dependencies from referers or crawling static files, but the technique is obviously only applicable to resources generated in push-aware code.
• On the relation between WAI application APIs: I've come to believe that HTTP2Application should be a superset (feature-wise) of Application, even if this means it has some vestigial features. This way, frameworks (e.g. Yesod) can be implemented against HTTP2Application directly, without losing access to HTTP1_-only features. Otherwise, Yesod would have to target *both_ APIs separately, which would be cumbersome. The original HTTP2Application didn't have this property (it dropped the non-streaming top-level constructors).

[davean]

• I originally had push exposed via an IO action to enqueue a promise,
  to match the semantics of the protocol more closely: server pushes can be
  initiated by the server at any time, they needn't be at the beginning of a
  stream. I don't know of any use cases that require this extra flexibility,
  so I don't feel strongly about it; just recording the motivation.

This seems fairly important to me. Frequently I generate pages from a total
model after some basic validation. I believe moving this to the start of
the request would require storage for arbitrarily large responses or
running the model computation twice, once to generate the dependencies and
a second time to actually send it. At least if one were using the feature.

[kazu-yamamoto]

I have resumed this project.

After deep consideration, I want to try to extend Response to hold HTTP/2 specific data. This avoid defining HTTP2Application. The HTTP/1.1 handler can just ignore the HTTP/2 specific data if any.

I have created the push branch:

https://github.com/yesodweb/wai/tree/push

I will try to implement server-push stuff in the HTTP/2 handler with STM. And its users (such as Referer: base) would be created as middleware.

[kazu-yamamoto]

OK. I found a cleaner way.

The new code provides getHTTP2Data and setHTTP2Data through vault of Request. No change to Response nor Application.

Now I convinced that this is the best way to provide the push APIs.

[kazu-yamamoto]

I added the Middleware based on Referer: in wai-http2-extra. Now I convinced that this push API is the right way to go.

[kazu-yamamoto]

Warp 3.2.7 has been released.