Brotli compression is very slow

[nikolajlauridsen]

Brotli seems to be extremely slow to compress files, the Umbraco backoffice JS bundle takes a whole 10 seconds to be served when using Brotli. After I overwrote the RequestHelper to instead use gzip when available the time fell to 1-2 seconds.

I realize though that Brotli seems to do a better job at compressing the file (~400 KB vs ~600 KB for gzip), however, 10 seconds seems an unreasonably long time to compress the file, so I'm thinking it could perhaps be a bug somewhere?

I'm guessing at the end of the day it might not be a huge problem if the file is aggressively cached when running in production, but it still seems like something is off to me.

[Shazwazza]

Potentially we need to adjust the compression level here

Smidge/src/Smidge.Core/CompositeFiles/Compressor.cs

Line 30 in b0383c3

compressedStream = new BrotliStream(ms, CompressionLevel.Optimal);

I was just reading about compression on dynamic resources here https://expeditedsecurity.com/blog/nginx-brotli/

And looks like the different levels of compression will drastically change the time it takes so I think we just need to find the right trade off.

[bergmania]

And looks like the different levels of compression will drastically change the time it takes so I think we just need to find the right trade off.

Sounds like a configuration option is valuable here then :)

[PhyxionNL]

Brotli is slow to compress at higher levels, while decompression is faster. Some benchmarks are here: https://quixdb.github.io/squash-benchmark/#results-table E: site seems bugged. Here's an alternative benchmark; https://community.centminmod.com/data/attachment-files/2019/04/7885_compress-benchmarks-010.png

But these files will be cached on disk (unless you use Smidge.InMemory), so it shouldn't matter too much. It's just a one time thing that happens.

[nikolajlauridsen]

Ah I see, that's some good insight, thanks.

The issue really is that we're using InMemory, and sometimes people will also use timestamp cache busting to test bundled/minified assets, so this lends to a quite bad experience for them.

For now we've made Brotli the lowest priority, however as @bergmania mentioned, the compression level would be a wonderful thing to be able to configure, that way we could reduce the compression level to something that would be a reasonable compromise. We could probably even do it depending on the cache busting, so if it's timestamp we could lower it more than if it was version string, or something similar.

[PhyxionNL]

Yes, configurable compression level would be useful and the best solution I think.

@Shazwazza What would you think about adding CompressionOptions? It'd probably best to be able to configure all three separately (Deflate, Gzip and Brotli) and adding these to SmidgeOptions as well.

[Shazwazza]

Sounds reasonable. But I wonder based on what @nikolajlauridsen says, if these would need to be configurable per bundle? Just like the cache buster is configurable per bundle. i.e. Global and Bundle level like many of the other options.

We need to find out what the 'ideal' level for brotli is and make that the default. I'm unsure if 'lowest' is the best. IMO the 'best' will be the setting that is faster and smaller than gzip.

[PhyxionNL]

I personally prefer a higher compression as default as decompression is faster and also has less data to transfer. Usually the static assets will be saved anyway, so it shouldn't matter too much. Although we could go with something lower as a default, such as 6 (which is a decent compromise between speed and compression).

Per bundle might be a good idea, there's already a CompressResult in bundle options anyway so it might make sense to have it configured there.

[willdean]

This recent article https://devblogs.microsoft.com/dotnet/performance_improvements_in_net_7/ has a bit about how the previous .NET defaults for Brotli were very very slow and how they're changing in v7.

The article is enormous, but search for Brotli and you'll find it.