Compression dictionary transport

[horo-t]

Introduction

If the browser and server share a dictionary, using the dictionary for compression can significantly reduce the amount of data transferred. This is especially useful for incremental library updates, or when browsing multiple pages that have a lot of common parts in their files.

This proposal adds support for using designated previous responses, as an external dictionary for Brotli-compressing HTTP responses.

HTTP Content-Encoding is extended with a new encoding type and support for allowing responses to be used as dictionaries for future requests. All actual header values and names still TBD:

• Server responds to a request for a cacheable resource with an bikeshed-use-as-dictionary: <scope/path>.
• The client will store a SHA-256 hash of the uncompressed response and the applicable scope for the resource with the cached response to identify it as a dictionary.
• On future requests, the client will match a request against an available dictionary with the best-matching scope. If a dictionary is available for a given request, the client will add sbr to the Accept-Encoding request header as well as a sec-bikeshed-available-dictionary: <SHA-256> header with the hash of the best available dictionary.
• If the server has a compressed version of the request URL with the matching dictionary, it serves the dictonary-compressed response with Content-Encoding: sbr and Vary: Accept-Encoding,sec-bikeshed-available-dictionary.

Read the complete Explainer.

Feedback

I welcome feedback in this thread, but encourage you to file bugs against the Explainer.

[pmeenan]

On the Chrome side we are getting ready to start experimenting and are having discussions with various sites and CDN's to work through any issues with the proposal in the explainer and start navigating it through the various working groups that it will impact.

I think we're at the point where it should move into WICG incubation to continue the discussions in public and coordinate the different standards efforts (likely to touch the IETF HTTP working group with a protocol-level RFC and the WHATWG fetch spec with browser-specific CORS, security and privacy considerations).

@horo-t if you can edit labels for the issue, can you tag it with the Protocols label?

[horo-t]

I think I don't have the permission to tag it with the label.

@travisleithead
Could you please add the label "Category: protocols"? Thank you.

[dangayle]

Hi. I am the lead performance engineer at Crate and Barrel. We'd like to express our interest in this. We're a large e-commerce brand who ships a ton of static JS files who would see a lot of benefit from it. We have a bi-weekly release schedule with versioned assets.

We don't have the people power to contribute meaningfully to the spec, but we would like to be a part of the discussion as as a test candidate and early adopter.

https://www.webpagetest.org/result/230323_BiDcTM_98R/

[valiantvu]

Pinterest would be interested in testing this out. We've spent a significant amount of time in tackling the amount of JS we're loading, which have yielded some great results, but in the end we're still sending an immense amount of code. We have quite a lot of repeat users and also deploy multiple times a day, so being able to send just the diff code would be an amazing improvement for at least our core & casual users. Based on some initial discussions, our traffic and platform teams think this could be worked into our build/serving systems.

[kurtextrem]

At Jochen Schweizer mydays, we deploy multiple new bundles per week as part of our micro-frontend architecture. Thus, we want to express interest in this proposal and would benefit from this feature as new bundles have minor changes compared to the previous bundle.

[yoavweiss]

@dangayle @valiantvu @kurtextrem - thank you for your support!! I believe that's enough for us to move things over to the WICG org.

[marcoscaceres]

Now at https://github.com/wicG/compression-dictionary-transport