Don't send link headers for content that is likely to be cached

[michaeldyrynda]

Using this middleware currently pushes resources on every single page load, which is additional bandwidth, and defeats the purpose of using it in the first place. Although the client may cancel the push request once it determines it already has the content being pushed, but this is after-the-fact i.e. once the content has already started being sent.

It's not possible to know for sure that content has been cached, but it is possible to take a "best guess" approach by setting a cookie on the request.

You can then assume that if the visitor has the cookie, you won't need to add the link headers.

If the visitor doesn't have the cookie, set it and send the link headers, if the visitor does have the cookie, we can assume that the content was already cached on the client side and skip sending the link headers.

The simplest way of doing this would be to set a cookie with an expiration of say one week. The client will handle cases where the CSS has changed during this time by falling back to normal HTTP requests, fetching the updated CSS, and subsequently caching it.

I've submitted the issue here as a placeholder to flag this, but will work on a PR for you to take a look at later on.

[JacobBennett]

Thanks Michael! Looking forward to the PR :)

Sent from GH.

[JacobBennett]

Would be interested to read the spec on this, and how it is being handled elsewhere. Do you have any resources that you used when looking around on this?

[michaeldyrynda]

Only the conversations I had with @bradleyfalzon about it; he implemented a similar thing in Go for a project he was toying with.

It makes sense, though. You don't want to push bytes to the client that aren't needed, only for them to be rejected. Brad was saying that I think CloudFlare and S3 were using a similar cookie-based approach.

[bradleyfalzon]

No CF/Apache use Link headers approach, the Cookie thing was something else.

[JacobBennett]

https://twitter.com/simevidas/status/760892541654949888

is Nopush the solution here, so it would just preload?

Reading this article gives some insights

https://blog.yoav.ws/being_pushy/

[GrahamCampbell]

Ideal situation would be to use machine learning to learn which files change more often and push them.

[GrahamCampbell]

An easier more immediate thing that would be easy to do is keep track of what files were already pushed on a session, and don't push them for the next request.

[bradleyfalzon]

@JacobBennett nopush would simply tell the web browser there is a resource it probably needs and to fetch it, negating the benefits of h2push.

@GrahamCampbell in theory is right on his second point, but to be effective under any reasonable amount of requests would likely consume too much RAM (each resource needs to be tracked for each session). You'd want to bound the memory used, and this could still push more resources than required. For many sites it'd be fine.

I'm of the opinion, unless you know a resource is required, you shouldn't push it to optimise performance, you don't know what bandwidth or battery is available to the client.

The cookie method is pretty straight forward, if you already have cookies with expire times in the far future (even just a couple days based on old Yahoo research), and someone browses to your website without cookies - it's almost certain they don't have resources and you can safely push those that you know they will need. This is a simple and cheap strategy that is almost universally applicable.

Also, be careful about which resources you're pushing, the client knows best which resources it needs first, so often it's just better to push the resources that really matter, essentially CSS and perhaps fonts (but fonts are harder - which font type does the browser support?).

[JacobBennett]

Just finished reading This Paper on the topic and I'm now more confused than ever on what this Middleware should be doing. This spec still seems very much under development and testing and as it matures I'm sure that this Middleware will have to change to accommodate best practices.

[joruro]

@JacobBennett Since at this moment is too soon to make a decision, why not you leave the responsible of choosing which files types to push and the push status to the developer side? For instance, using a config.

[JacobBennett]

@joruro this exists over here although I may eventually add the ability to push additional resources with a global helper.