Compress html documents when sending them to the user

[bendem]

Html documents served by docs.rs are not compressed resulting in large bandwidth waste on pages for complex types.

See for example the iterator page as discussed here: rust-lang/rust#55900 (comment)

[GuillaumeGomez]

I wondered for a long time if we should handle compression directly inside rustdoc. That could be a good time to talk about it as well...

[pickfire]

If the docs are compressed locally, rustup doc --std probably needs some changes such as starting up a server instead of just opening the file, I think it affects search as well.

[GuillaumeGomez]

It would indeed. Which is why we never did.

[pickfire]

@GuillaumeGomez The current possible solution may be minifying the html resource which can probably shave like ~5% off while keeping the same output without much changes. What do you think?

Of course, compared to compressing which could save like 90% off it's small. Just compressing individual files already get 10% of original size, think compressing after archiving into tarball.

.rw-r--r-- 358k pickfire 25 Dec 00:00 iterator.rs.html
.rw-r--r--  26k pickfire 25 Dec 00:00 iterator.rs.html.bz2
.rw-r--r--  43k pickfire 25 Dec 00:00 iterator.rs.html.gz
.rw-r--r--  25k pickfire 25 Dec 00:00 iterator.rs.html.xz
.rw-r--r--  36k pickfire 25 Dec 00:00 iterator.rs.html.zst

[GuillaumeGomez]

If the web browsers can still work with local compressed files, then I'm totally going to add it.

[syphar]

NGINX & CloudFront are serving compressed content when the browser supports it.

So I assume this issue is solved.

Please feel free to reopen if not.

[pickfire]

The local docs are not compressed, it took up 635 MB for the current stable, half the size of the whole toolchain (1.2 GB). I think we can reopen this, but one issue is browser does not work with locally compressed files.

[syphar]

This repo here is purely about https://docs.rs .

"local docs" sounds like this is about rustdoc itself (when it's about crate docs), which lives in https://github.com/rust-lang/rust , or the std docs, which live there too.