Improve recognition of text MIME types for compression

[palant]

Description

If you look at the list of MIME types potentially produced by the mime_guess package, the list in compression::TEXT_MIME_TYPES isn’t really great for recognizing text MIME types. This implements the following changes:

• Generic recognition for MIME types starting with text/ or ending in +xml/+json
• Faster testing of the remaining MIME type list by using a HashSet
• Recognition no longer confused by charset parameter (a bug I mentioned in Apply the usual post-processing to error responses for consistency #359)
• Adjusted font MIME types to the ones actually produced by mime_guess and removed the outdated application/x-javascript type

Note that lazy_static isn’t a new dependency, we already have it as a transitive dependency via tracing-subscriber.

How Has This Been Tested?

Functional testing on Linux looks fine, and Rust files (text/x-rust) are now being compressed as well.

[semanticdiff-com]

Review changes with SemanticDiff.

Analyzed 1 of 3 files.

Overall, the semantic diff is 34% smaller than the GitHub diff.

	Filename	Status
❔	Cargo.lock	Unsupported file format
❔	Cargo.toml	Unsupported file format
✔️	src/compression.rs	33.16% smaller

[palant]

Unfortunately, this was public, so this is a breaking change.

[joseluisq]

Let's keep the comment open for mentioning this breaking change in the next release.

[palant]

Benchmark results show a clear improvement on small text files (going down in the noise on large ones). Seemingly no real change on the directory index (text/html was listed first, so recognizing this MIME type was faster).

[joseluisq]

Thanks!

[joseluisq]

Let's keep the comment open for mentioning this breaking change in the next release.

[palant]

This caused a significant performance regression on directory listings for some reason.

[palant]

Ah, I see – not actually a regression. Previously, we wouldn’t compress directory listings because the original logic wouldn’t recognize text/html; encoding=utf-8 as a text format. And now that we do compress directory listings, performance naturally went down.

[joseluisq]

Ah, I see – not actually a regression. Previously, we wouldn’t compress directory listings because the original logic wouldn’t recognize text/html; encoding=utf-8 as a text format. And now that we do compress directory listings, performance naturally went down.

It would be expected, but is there a big margin?

[palant]

It would be expected, but is there a big margin?

Sure, compression takes time (gzip in particular which is what vegeta is testing). So directory listing tests went up by almost 50%.