cache-aware server-push

[kazuho]

To optimize the responsiveness of a website, it is desirable to push assets (e.g. CSS and/or JavaScript files) that block the page rendering process when and only when such files do not already exist within the client's cache. However, the specifications do not define a way to determine whether if a resource is already cached.

This issue proposes of using a cookie containing a probablistic data set as an approximation for determining the cache state of the client, and using it for restricting server-push - i.e. only push the asset files that are unlikely to be cached.

Basic Design

• use a single cookie that stores a Golomb coded set as to mark if the assets are cached
• key for the set will be calculated as key(filename)=sha1(filename + etag(filename))
• for each HTTP2 connection, the server maintains the value of the set
  • the set is initialized as empty
  • when receiving the cookie, the value of the set is set the the received value
  • when starting to send a response, for that response mark the set using a key calculated using the rule above, and send set-cookie header containing the value of the set
  • when application logic triggers a server-push, check if the to-be-pushed content is marked within the set, and if marked, ignore the trigger. If not, push the content to the client at the same time updating the value of the set using the rule above.
• server may also maintain a set that maps all the existing files, and use the set to clear the keys of the incoming set that do not exist (i.e. means that the corresponding file has either been removed or updated), before copying the value to the internally maintained value

By applying this design for restricting server push, the only drawback of using server push will become when the cookie on the client expires before the asset cached in the client's cache expires. In other words, the design will work effectively if such case is unlikely.

Notes

Since HTTP2 multiplexes the HTTP requests over a single stream, it has become possible for the web server to expect in which order the client receives the responses (i.e. we can safely say that: if the client has received the headers of response A, then it must have received the headers of response B that has been sent prior to A). The design takes advantage of the property.

A good read on Golomb coded set is this blogpost. The only issue the doc has is how M of Golomb coded is calculated; theoretically it should use median instead mean value.

[kazuho]

@igrigorik @tatsuhiro-t @summerwind @Jxck You might be interested in this.

[Jxck]

looks so nice ! but I wanna ask some point of view.

1, using cookie or frame (http1/http2)

cookies used lots of purpose and it has limited to under 4K.
so it seems better to add new semantics to other properties, like new header or so.
in this idea, browser adds cache information to http request, so there are no problem to adding new X- headers.

but I wonder this is adding semantics to http1 or http2.
if adding to http1, that means adding new http header,
but if adding to http2, adding new http2 Frame or setting seems better.

and server push is only for http2. and http1 server could do nothing for receive cached status.
so I think its better to add Frame or settings to http2, instead of adding data to cookies.

2, way to ask client without request

it's just idea, but there are no way to ASK browser this status ?
for example, in case of offline cached application using service worker,
server wanna push content if that's newer than service worker caches.
in that case, server wanna ask cache status without browser request.

3, security consideration

in http1, Etags are used to tracking user.
so I wonder this Idea may increase information for tracking user or not.
(ex, serve different name content to each user every time, and browser sends this information. could server gets the order of page user visited ?)

[kazuho]

@Jxck Thank you for your comments.

1, using cookie or frame (http1/http2)

cookies used lots of purpose and it has limited to under 4K.
so it seems better to add new semantics to other properties, like new header or so.

No. It need not be.

As discussed in the blog post linked above, the theoretical minimum size of a probability data set like Golomb coded set is N * log2(1/P) bits where N is the number of files to be potentially be pushed, and P being the probability of false positive, and Golomb coded set is pretty near to the minimum.

So for example, if you have 100 CSS or JavaScript files on your website (N=100), and want the probability of false positive to be 1%, the size of the coded set is estimated to be slightly above 83 bytes (100 * log2(1/100) bits). Of course we need to encode the value to fit in the characters permitted as HTTP header values, but anyways it is far below the 4K limit.

EDIT. FWIW http://browsercookielimits.squawky.net says that there is no per-domain cookie size limit for Chrome and Firefox. Safari has a stricter limit (8K per domain) and we might be careful on using this feature for the web browser once the new version with HTTP/2 support gets shipped.

2, way to ask client without request

it's just idea, but there are no way to ASK browser this status ?

Not that I am aware of. I doubt if there is such a way considering the fact that we need to include the status as part of the first HTTP request that is sent by the client.

3, security consideration

in http1, Etags are used to tracking user.
so I wonder this Idea may increase information for tracking user or not.

That is a good question.

I believe that this implementation (by using cookies) does not impose any new privacy issues.

As explained, I believe that this feature can be implemented using Cookies. It is also true that this feature must be implemented by using an information not bound to a specific URI (like etag). In other words it does not abuse etag.

[igrigorik]

A few thoughts as I'm reading this:

1. Cookies are per-origin, so each origin would have to replicate such functionality.
2. Many existing "static hosting providers" are "cookieless domains".
3. Having a cookie indicating expected resource TTL does not indicate that the resource is actually in cache; there is no way to traverse the HTTP cache either.

(3) in particular I think might be an issue in practice. We do know that caches get cleared often (although not sure that we have a great understanding of why that is; something we're investigating in Chrome), and that might render all of this not that useful.

FWIW, I think it might be worth exploring a SW-interop story first.. With SW you can manage your own caches and you can append arbitrary headers, which addresses all of the above. The story would then become: drop this SW script on your site and deploy this smart server on the other end, the two of them will talk to each other to figure out how and when resources are updated, when they're pushed, etc.

[kazuho]

@igrigorik Thank you for your comments.

(1) Cookies are per-origin, so each origin would have to replicate such functionality.
(2) Many existing "static hosting providers" are "cookieless domains".

I am considering of implementing this feature within H2O to miminize the users' burden on using the feature.

Once this issue gets implemented, web applications do not need to take care of Cookies. All they need to do is to say I want these assets to be pushed (presumably via the Link header), and the web server (by looking at the Cookie and the etags of the pushed assets) determine whether they should in fact be pushed or cancelled.

(3) in particular I think might be an issue in practice. We do know that caches get cleared often (although not sure that we have a great understanding of why that is; something we're investigating in Chrome), and that might render all of this not that useful.

I agree. I did not know that Chrome has such issues. Thank you for the information.

But all the side effect of such false-positive (i.e. cookies incorrectly says that it is cached) is that it disables the server-push for the file. The client can simply pull the file in such case (which means that it is self-reparing as well).

So even if such case seem to be somewhat common (e.g. 10% of revisiting users suffer from the issue), we can enable server push making 90% of the revisting users and all new users happy, without causing any negative impact on the user-experience of the users that do suffer from the issue.

FWIW, I think it might be worth exploring a SW-interop story first.. With SW you can manage your own caches and you can append arbitrary headers, which addresses all of the above. The story would then become: drop this SW script on your site and deploy this smart server on the other end, the two of them will talk to each other to figure out how and when resources are updated, when they're pushed, etc.

Thank you for the suggestion.

While I agree that using ServiceWorker when possible would give you more precise control, my understanding is that in case of speeding up a web-page load there is a chiken-egg problem.

My understanding is that not until the browser start rendering a webpage it is possible for the ServiceWorker to communicate with the script on that page (and for the webpage to communicate with the ServiceWorker the webpage would need to load certain scripts).

The design described on this issue covers such use cases.

Considering the fact that the approach proposed in this issue is fairly simple, I am thinking that the best way forward would be to implement it first, and learn from it how we can improve.

[igrigorik]

I agree. I did not know that Chrome has such issues. Thank you for the information.

To clarify, it's not just Chrome... same pattern repeats across all browsers: https://code.facebook.com/posts/964122680272229/web-performance-cache-efficiency-exercise/

My understanding is that not until the browser start rendering a webpage it is possible for the ServiceWorker to communicate with the script on that page (and for the webpage to communicate with the ServiceWorker the webpage would need to load certain scripts).

No, once the SW is registered it is woken up the moment the navigation request is made. At this point the SW has full control of whether it wants to dispatch the nav request, serve from cache, etc.. Same logic applies to all subresources as well.

[kazuho]

@igrigorik

To clarify, it's not just Chrome... same pattern repeats across all browsers: https://code.facebook.com/posts/964122680272229/web-performance-cache-efficiency-exercise/

Thank you for the clarification and the link! With the numbers it is possible to estimate how much the benefits of implementing cache-aware server-push would be.

No, once the SW is registered it is woken up the moment the navigation request is made. At this point the SW has full control of whether it wants to dispatch the nav request, serve from cache, etc.. Same logic applies to all subresources as well.

Thank you for the clarification. Looking back to the SW docs, apparently I have missed one of the key aspects of the specification.

Now I agree not only using ServiceWorker is possible but also that it is optimal.

Presumably, we could also push the script file containing the ServiceWorker-related code by detecting the absense of the modifications to the headers done by ServiceWorker.

[kazuho]

The feature should be limited to HTTP2 over TLS (not any HTTP2), since in case of h2c proxies may scramble the response order (pointed out by @tatsuhiro-t).

[kazuho]

I have done some initial tests using a prototype I have built using Firefox, and the fact is that it is hard to guarantee the order in which the cookie in the responses are recognized by the web browser even though we can control the order in which the responses are received by the browser in case of h2.

In other words, it is essential to set the cookie using client-side logic using JavaScript, and I believe that can be done without using ServiceWorker as well (with the exception that you would not be able to remove entries that have been purged from cache).

[igrigorik]

@kazuho can you elaborate on why the order is important? Not sure I understand what the issue is there.

[kazuho]

@igrigorik Of course!

The fundamental requirement of cache-aware server push is that we need to have a fingerprint of what is being cached. And since it is a fingerprint, a single value (essentially a bit-vector) contains information of all the asset files being cached.

Consider we are sending two asset files (named A and B with A being the first and B being the second). Also consider that the corresponding bit within the Bloom filter for A is 1, and for B is 9.

In this case, we would be sending two responses:

# response for A
HTTP/2 200 OK
Set-Cookie: fingerprint=0100000000

[content-of-A]
# response for B
HTTP/2 200 OK
Set-Cookie: fingerprint=0100000001

[content-of-B]

As you can see, at the moment we have sent A, we should only set fingerprint for A (but not B), since B is not delivered yet. And then after sending B then we should set fingerprint to contain both A and B.
(off topic: it would have been desirable to set the cookie in trailer)

The assumption I had was that since h2 is a stream, we could guarantee that the set-cookie headers are recognized by the browser in the order they are sent, under the premise that they are in the same order as they appear in HTML.

But it seems that (at least for Firefox) the expectation seems to have been wrong, it was not always the case that the cookie contained in the last response was processed last.

The fact means that we cannot rely on the server sending Set-Cookie header to set the correct fingerprint; we need to use either XHR or SW and in their callbacks (or promises) update the fingerprint (stored in the cookie) to appropriate value.

[igrigorik]

@kazuho would it be possible to emit just one Set-Cookie per "push block" instead ? From client's perspective the per-resource set-cookie is overhead; you just need to emit one such directive per "push block"... if you know what you need to push on the server ahead of time, seems like that shouldn't be too hard to implement?

[kazuho]

@igrigorik Thank you to you and Patrick for the suggestions at the workshop.

I think I have found a way to work around the issue without using JavaScript.

Until now, I have tried to update the cookie for every asset being sent. But as you pointed out, we could do it other ways.

I am now considering of updating the cookie in the parent stream for all the pushed sub-streams at once. In case of the example I showed, I am thinking of sending the Set-Cookie header associated to the response of index.html.

It is true that such an approach might cause false-positives (i.e. resource that has not been cached might get marked as such if the connection terminates early).

But is that an issue? I do not think so. All what happens in case of having a false-positive is that push does not occur for that asset; it simply gets pulled by the browser.

OTOH this approach is good in the fact that it prevents false-negatives. In case of the original approach that included Set-Cookie headers in the pushed responses, there was a possibility that an asset being pushed does not get marked, when the asset was never used by the web browser.

This would become a real problem when server administrator mis-configures what to push. If an asset that is never actually used (from HTML) gets pushed, in the original approach the Set-Cookie header of the pushed resource will be ignored in case of Firefox (FWIW my understanding is that Chrome recognizes the Set-Cookie header in this case as well), and it may trigger that resource to get pushed every time the user visits the web page (because the cookie with the corresponding bit set never gets recognized).

As said, this drawback in the original approach (BTW thanks to @Jxck for pointing it out on the chat) will disappear if we set the cookie in the parent stream, which to me seems like a really good thing.

[igrigorik]

I am now considering of updating the cookie in the parent stream for all the pushed sub-streams at once. In case of the example I showed, I am thinking of sending the Set-Cookie header associated to the response of index.html.

Hmm, wouldn't that open up the reverse race case? As in, when is the cookie value updated when its sent on main doc? If its applied when headers are parsed then won't you falsely report the sent cookie value on all subresources?

[kazuho]

Hmm, wouldn't that open up the reverse race case? As in, when is the cookie value updated when its sent on main doc? If its applied when headers are parsed then won't you falsely report the sent cookie value on all subresources?

Yes, but IMO that is not so big problem for two reasons:

• 1. all the side-effect of such case is that the client would need to pull that resource (which is no worse than the situation without push)
• 2. talking of how H2O is implemented, pushed streams are given priority value of 257 internally, which means that they would be sent before anything else (even before the <HEAD> of the HTML that triggered the push). It is logical to do so, considering the fact that the resources we need to push are those that block rendering. And the fact means that it is highly likely that at least some of the pushed data will reach the client side within the same packet that includes the response HEADER frame for the parent stream (that contains the Set-Cookie header).

Regarding the 2nd point, I think reducing the bits sent over the network is prior to reducing RTT in general. In other words, I prefer refraining from pushing assets that might exist in cache.

EDIT. If this thing goes well it is definitely true that we should also consider pushing low-priority assets (e.g. images) as well.

[igrigorik]

all the side-effect of such case is that the client would need to pull that resource (which is no worse than the situation without push)

In your tests, when do you see the cookie header being applied? Is the behavior consistent across browsers?

talking of how H2O is implemented, pushed streams are given priority value of 257 internally, which means that they would be sent before anything else (even before the of the HTML that triggered the push). It is logical to do so, considering the fact that the resources we need to push are those that block rendering.

Except for the case where some of the blocking resources reside on a different origin - e.g. a font CDN, content/analytics scripts, and so on. FWIW, I don't think you should unconditionally place push resources ahead of the markup... Also, I may want to push an image to reduce latency, but I certainly don't want it blocking HTML or CSS.

[tinganho]

when application logic triggers a server-push, check if the to-be-pushed content is marked within the set, and if marked, ignore the trigger. If not, push the content to the client at the same time updating the value of the set using the rule above.

Wouldn't this require a server push from the application server for every request? (Assuming h20 acts as reverse proxy).

[tinganho]

Also if there is SW support. Probably there won't be a second same page request, because it can be cached? So no need for cache-aware server push at least not for the same page? And if navigating to an another page the client can hijack the page request and send info about what need to be push(or what is already cached) and the application server just push what needs to be pushed.

So in this proposal application server handles all the server push logic. But it would be great if the application server can delegate the push to a reverse proxy instead of doing the server push itself. To minimize memory footprint and connection time of the application server and bandwidth between the application server and the reverse proxy. Something along the line with a custom HTTP header X-Push: /styles.css, /page1.css and then the reverse-proxy fetches the content, caches it and pushes it to the client.

[kazuho]

@igrigorik

all the side-effect of such case is that the client would need to pull that resource (which is no worse than the situation without push)
In your tests, when do you see the cookie header being applied? Is the behavior consistent across browsers?

Now that the server is updating the cookies only in the responses to requests being pulled, on both Chrome and Firefox I see that they are applied in the order the server sends them (over h2).

FWIW, I don't think you should unconditionally place push resources ahead of the markup... Also, I may want to push an image to reduce latency, but I certainly don't want it blocking HTML or CSS.

Agreed. However there are issues that need to be sought out if you want to push low-priority resources.

First is the overhead of the PUSH_PROMISE frames. If we are going to push resources, we want to make sure that the web browser does not pull the same resource. To fulfill such requirement, we need to send PUSH_PROMISE frames for all the responses that need to be pushed before sending the response to the pull request. This may take up substantial portion of the bandwidth available in the first RTT, in case which may lead to deterioration of the first-paint time.

Second is the problem of weight. What is the appropriate weight for such non-blocking resources? Should we schedule them as dependents of the parent stream, or should we schedule as siblings to the parent stream with lower weight value?

Considering these issues, I think we should better start off by pushing only the blocking assets at first, and then determine how we should support the non-blocking ones.

[kazuho]

@tinganho

So in this proposal application server handles all the server push logic. But it would be great if the application server can delegate the push to a reverse proxy instead of doing the server push itself. To minimize memory footprint and connection time of the application server and bandwidth between the application server and the reverse proxy. Something along the line with a custom HTTP header X-Push: /styles.css, /page1.css and then the reverse-proxy fetches the content, caches it and pushes it to the client.

Thank you for your comments. Actually, what you propose is what H2O is doing now.

H2O and the reverse proxy of nghttp2 both recognize Link rel="preload" header, fetches the linked resource on behalf of the web browser, and pushes them to the browser (please see #133 #137 https://nghttp2.org/blog/2015/02/10/nghttp2-dot-org-enabled-http2-server-push/).

This issues is about cancelling such pushes within the server in case the browser already has the resource in its cache.

[tinganho]

@kazuho yes and my proposal is to let the SW handle that job. But I think it would be more ideal to use an HTTP header as oppose to an html link tag. It's much easier to append an HTTP header than to inject an HTML. Although link tag might be suitable for static web pages.

[kazuho]

@tinganho

yes and my proposal is to let the SW handle that job.

I think we have a common understanding that SW should be used whenever possible. However, SW cannot be started until the first page (and the SW script) is loaded, which we would want to speed up as well by using server push.

So the plan is to first implement the cookie-based version, and then implement the client-side code (using SW) that modifies the cookie.

But I think it would be more ideal to use an HTTP header as oppose to an html link tag. It's much easier to append an HTTP header than to inject an HTML.

At the moment, only the headers are recognized.

[tinganho]

I think we have a common understanding that SW should be used whenever possible. However, SW cannot be started until the first page (and the SW script) is loaded, which we would want to speed up as well by using server push.

If your website is an application(not static HTML). This should be handled by the application server? If the application server don't receive data about what to push then it should push all by default, i.e. on the first page load it would push everything. On the next page load SW is installed and it can send data about what should be pushed or is already pushed. And the application server just push what SW decides to push.

[igrigorik]

@tinganho as much as I'm a fan of SW, I don't think we necessarily need or want to force it on every H2-enabled site -- e.g. not all browsers support SW yet, whereas push+cookies is a sensible cross-browser strategy that requires minimal work. There is room for both, and you may decide that your application should use the SW approach.. but for wider deployment cookies is a good strategy.

@kazuho inline..

Agreed. However there are issues that need to be sought out if you want to push low-priority resources.
First is the overhead of the PUSH_PROMISE frames. If we are going to push resources, we want to make sure that the web browser does not pull the same resource. To fulfill such requirement, we need to send PUSH_PROMISE frames for all the responses that need to be pushed before sending the response to the pull request. This may take up substantial portion of the bandwidth available in the first RTT, in case which may lead to deterioration of the first-paint time.

Perhaps.. we only need to send compressed headers, so unless you're pushing a very large number of resources I wouldn't expect that to be the common problem case.

Second is the problem of weight. What is the appropriate weight for such non-blocking resources? Should we schedule them as dependents of the parent stream, or should we schedule as siblings to the parent stream with lower weight value?

As one strategy, you can use content-type to determine the dependencies and weights. E.g. replicating the current FF implementation you'd put CSS and JS into same "leaders" group as the HTML, but images would go into "followers" which depend on "leaders".

Note that in Preload spec we're still hashing out the right plumbing to surface "fetch this resource as X" semantics.. once we have that nailed it'll be communicated via the Link header as well.

Considering these issues, I think we should better start off by pushing only the blocking assets at first, and then determine how we should support the non-blocking ones.

Sure. I'd just put a disclaimer to folks experimenting with this stuff that any pushed resource preempts all other response bytes (which, to me, still doesn't smell right ;-))

[kazuho]

@igrigorik

Second is the problem of weight. What is the appropriate weight for such non-blocking resources? Should we schedule them as dependents of the parent stream, or should we schedule as siblings to the parent stream with lower weight value?

As one strategy, you can use content-type to determine the dependencies and weights. E.g. replicating the current FF implementation you'd put CSS and JS into same "leaders" group as the HTML, but images would go into "followers" which depend on "leaders".

Regarding how to schedule CSS and JS, they need to block HTML (which is what Firefox does; please refer to https://speakerdeck.com/summerwind/2-prioritization?slide=18; and is essential for responsiveness; see http://blog.kazuhooku.com/2015/06/http2-and-h2o-improves-user-experience.html).

OTOH, how images being pushed should be prioritized is a tough question.

We have two requirements:

• pushing the images should not delay the browser from rendering the HTML
  • in other words, body of the images should not be sent before <IMG> tags that link to them
• pushed images should be given same (or greater) weight than the images being pulled

Let's go back to the Firefox's dependency tree to consider the first requirement. The browser schedules images as siblings to HTML (with lower weight, see the link above). The approach makes sense in case the images are pulled; they start being pulled after <IMG> tags arrive at the client. But in case of pushing the images, we should better not do the same, because if we did, images will begin being interleaved not after <IMG> tags are sent, but from the very first frames being sent. If we did so, it is likely that the responsiveness would become worse. This is why I think that HTML should block the images if we are going to push the images.

The second requirement is hard to fulfill without the cooperation of the client. Ideally, the web browsers should recognize how the pushed images are scheduled (by examining the dependency tree), reprioritize them using the browsers logic, and send PRIORITY frames back to the server. But I am not sure if we could expect all the web browsers to do such thing correctly.

Anyways, I am not saying that we should not push images, but am trying to determine what the ideal approach in case of doing so is, as well as the obstacles.

Note that in Preload spec we're still hashing out the right plumbing to surface "fetch this resource as X" semantics.. once we have that nailed it'll be communicated via the Link header as well.

That would be great! My two cents go into categorizing the resources to blocking (e.g. CSS), non-blocking (e.g. images), deferred (e.g. <script> at the end of the body). As discussed in this issue, the prioritization logic of HTTP2 is far too complicated to be scheduled without looking into the dependency tree.

Sure. I'd just put a disclaimer to folks experimenting with this stuff that any pushed resource preempts all other response bytes (which, to me, still doesn't smell right ;-))

Yeah! I understand. I agree that the fact that I believe it is hard to push images (for better response-time) is hard does not mean that H2O should always give pushed resources the highest priority. I expect to add flags to specify the priority (possibly by the three-level approach) to the mimemap definition as part of #436.

[igrigorik]

Let's go back to the Firefox's dependency tree to consider the first requirement. The browser schedules images as siblings to HTML (with lower weight, see the link above). The approach makes sense in case the images are pulled; they start being pulled after tags arrive at the client.

My understanding was that images are put into "followers", which are dependent on "leaders" (HTML, CSS, JS), see: http://bitsup.blogspot.se/2015/01/http2-dependency-priorities-in-firefox.html

[kazuho]

@igrigorik

My understanding was that images are put into "followers", which are dependent on "leaders" (HTML, CSS, JS), see: http://bitsup.blogspot.se/2015/01/http2-dependency-priorities-in-firefox.html

I see. The truth is that HTML is scheduled as followers.

[kazuho]

#432 and #436 has been merged to master.

And now, the internals related to prioritization are as follows:

• every mime-type has a priority assigned: either NORMAL or BLOCKING
  • CSS and JavaScript files are considered BLOCKING, others are considered NORMAL (mimemap.c line 141)
  • there is (yet) no configuration directive to adjust the priorities
• in case of server-push, BLOCKING assets are given weight=257 at the root level of the dependency tree, meaning that they would be given precedence to any other streams
• in case of pull requests, BLOCKING assets are also given weight=257 at the root level if http2-reprioritize-blocking-assets directive is being set and if the client is not using dependency-based prioritization (i.e. any web browser except Firefox) (see also #349)
• in all other cases, pushed streams are prioritized as dependent to the parent stream with weight=16 (ref: RFC 7540 section 5.3.5)
• we do not send PRIORITY frames in either case in which we set the internal weight of 257; these are the cases we are smarter than what the web browser thinks!

For cache-aware server-push, a configuration directive named http2-casper is provided (casper stands for cache-aware server pusher). The directive can be used either at global level or host level.

• if the value of the directive is set to OFF, cache-aware server push is not used (default behavior)
• if set to ON, cache-aware server push will be used with the default settings described right below
• if the value is a mapping, following attributes are recognized:
  • capacity-bits sets the size of the bloom filter; default is 13 (2**13 ~= 100 tracking assets with collision probability at 1%); casper will be disabled if the value of the attribute is set to zero
  • tracking-types set the types of the assets that need to be fingerprinted; default is blocking-assets which will limit tracking to the BLOCKING types. If set to all, all responses will be tracked

[kazuho]

@igrigorik The thing I forgot to mention when discussing about limiting cache-aware server push to blocking asset files only was the size of the fingerprint.

As discussed in #421 (comment) the size of the fingerprint (compressed using golomb coded set) is proportional to the number of assets being tracked; and therefore tracking only the files that matter the most is IMO a safer approach.

[kazuho]

ToDos related to cache-aware server push is as follows:

until 1.5:

• #493 do not emit access log in case server-push is cancelled by casper
• in access log, pushed responses must be distinguishable from pull requests
  • can be done by emitting the x-http2-pushed header (e.g. %{x-http2-push}o)
• #496 provide configuration directive to modify the priority (and the is_compressible flag) associated with the mime types
• #492 disable casper if proxy is used
  • casper relies on the assumption that the client will recognize the cookie in the order they are sent (on a single stream); however such assumption can only be met when there is an end-to-end TLS connection using h2

long-term:

• provide ServiceWorker-based implementation
• implement a way to create positive list of the fingerprint that can be used to evict the negative ones (i.e. flags corresponding to files being removed or altered) from the client-side cookie
  • note: this is only necessary for the cookie-based approach; when using SW we can build an accurate fingerprint on the client-side by iterating through the SW cache

[kazuho]

@mcmanus As shown in
HTTP/2 Deep Dive: Priority & Server Push p. 73 by @summerwind, for the streams being pushed Firefox sends PRIORITY frames with dependency=0 and weight=2.

Is there a reason why? IMO web browsers should assign to the pushed streams the same priority as if they were pulled (based on how they are used).

The reason I ask is as follows.

At the moment, H2O pushes blocking assets at weight=257 (257 is an internal value meaning send before anything else), while non-blocking assets are pushed using the default values as specified in the HTTP specification (please see this comment for details).

For non-blocking assets at least, I believe that we should better rely on the web browser to give them appropriate priorities. However, with the current implementation of Firefox, the pushed images will be given virtually no bandwidth compared to the images being pulled.

[qgy18]

If someone using ctrl + f5(win) or cmd + shift + r(mac) for disregarding any cache, should the server ignore the h2o_casper fingerprint?

[karantin2020]

Hello. If I enable http2-casper then assets do not push from server at all. They are downloaded in common way after web page loaded.
How to enable server push with http2-casper when use libh2o?
Thanks