Support for zstd in Deflater

[sandstrom]

Support for Content-Encoding value zstd in Rack::Deflater.

This algorithm is currently only supported by Chrome and Edge. Firefox has an open issue with little activity and Safari has "positive" as their standard position, but I could see no implementation work.

Perhaps it's too early to spend time on this, but wanted to raise the issue nonetheless.

Useful for things such as JSON payloads and other dynamic content that cannot be compressed ahead of time. The compression ratio is similar to gzip, but it's twice as fast to decompress and (I think) slightly quicker to compress too.

Resources

• https://caniuse.com/zstd
• https://lemire.me/blog/2021/06/30/compressing-json-gzip-vs-zstd/

[ioquatix]

Is there a standard library for providing this? I'd be okay with accepting a PR to implement this.

[sandstrom]

@ioquatix This is the one I know about:

https://github.com/SpringMT/zstd-ruby

[ioquatix]

I think the way I see this playing out:

1. We should define a standard interface for compression algorithms.
2. Define some way to specify the zstd as an option in Rack, e.g. require 'rack/deflater/zstd' or something.

I don't think rack should depend on another gem, but I have no problem exposing some standard mechanism that makes it extremely easy to pull in a zstd implementation and use it everywhere that makes sense.

[sandstrom]

We should define a standard interface for compression algorithms.

I don't think rack should depend on another gem, but I have no problem exposing some standard mechanism that makes it extremely easy to pull in a zstd implementation and use it everywhere that makes sense.

I agree, this makes a lot of sense for several reasons!

Could be coupled with Rack docs pointing out a few third-party gems, or just examples of how to implement it.

[jeremyevans]

To support this and potentially other compression algorithms, I think we should provide a Deflater option for custom deflaters:

use Rack::Deflater, deflaters: {"zstd"=>lambda{|headers, body| ...}}

The callable would return the deflated body. The default would be something like:

{
  "gzip"=>lambda do |headers, body|
     mtime = headers["last-modified"]
     mtime = Time.httpdate(mtime).to_i if mtime
     GzipStream.new(body, mtime, @sync)
  },
  "identity"=>lambda{|headers, body| body}
}

Not saying the implementation would use that (should probably keep the current behavior hardcoded for performance), but that would be the behavior.