Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Issue 15449 - CSS and JS files should be minified #2311

Closed
wants to merge 1 commit into from

Conversation

wilzbach
Copy link
Member

@wilzbach wilzbach commented Mar 30, 2018

A start anyhow:

> ll js
total 304K
drwxr-xr-x  2 seb users 4,0K Mar 20 12:22 ./
drwxr-xr-x 21 seb users 4,0K Mar 30 06:56 ../
-rw-r--r--  1 seb users 156K Feb  5 12:26 codemirror-compressed.js
-rw-r--r--  1 seb users 3,6K Feb  5 18:54 ddox.js
-rw-r--r--  1 seb users 3,0K Feb 27 23:16 dlang.js
-rw-r--r--  1 seb users  93K Feb  5 12:26 jquery-1.7.2.min.js
-rw-r--r--  1 seb users 5,8K Mar 20 12:22 listanchors.js
-rw-r--r--  1 seb users 1,6K Feb  5 12:26 platform-downloads.js
-rw-r--r--  1 seb users 3,8K Mar 13 11:56 run_examples.js
-rw-r--r--  1 seb users  16K Mar 13 11:56 run.js
-rw-r--r--  1 seb users 2,1K Feb  5 12:26 show_contributors.js

> ll web/js
drwxr-xr-x  2 seb users 4,0K Mar 30 06:56 ./
drwxr-xr-x 10 seb users 4,0K Mar 30 06:56 ../
-rw-r--r--  1 seb users 156K Mar 30 06:54 codemirror-compressed.js
-rw-r--r--  1 seb users 3,6K Mar 30 06:54 ddox.js
-rw-r--r--  1 seb users 3,0K Mar 30 06:54 dlang.js
-rw-r--r--  1 seb users  93K Mar 30 06:54 jquery-1.7.2.min.js
-rw-r--r--  1 seb users 5,8K Mar 30 06:54 listanchors.js
-rw-r--r--  1 seb users 1,6K Mar 30 06:54 platform-downloads.js
-rw-r--r--  1 seb users 3,8K Mar 30 06:54 run_examples.js
-rw-r--r--  1 seb users 7,7K Mar 30 06:56 run.js
-rw-r--r--  1 seb users 2,1K Mar 30 06:54 show_contributors.js

(the biggest chunks are jQuery + CodeMirror, but the other scripts are reduced by half.)

> ll css
total 64K
drwxr-xr-x  2 seb users 4,0K Mar 30 01:29 ./
drwxr-xr-x 21 seb users 4,0K Mar 30 06:56 ../
-rw-r--r--  1 seb users 7,9K Feb  5 12:26 codemirror.css
-rw-r--r--  1 seb users 2,8K Feb  5 12:26 ddox.css
-rw-r--r--  1 seb users 3,2K Mar 13 11:57 html-skeleton.css
-rw-r--r--  1 seb users  349 Feb  5 12:26 print.css
-rw-r--r--  1 seb users  35K Mar 30 01:29 style.css
> ll web/css
total 48K
drwxr-xr-x  2 seb users 4,0K Mar 30 06:54 ./
drwxr-xr-x 10 seb users 4,0K Mar 30 06:56 ../
-rw-r--r--  1 seb users 5,3K Mar 30 06:54 codemirror.css
-rw-r--r--  1 seb users 2,4K Mar 30 06:54 ddox.css
-rw-r--r--  1 seb users  287 Mar 30 06:54 print.css
-rw-r--r--  1 seb users  23K Mar 30 06:54 style.css

CC @JackStouffer

@CyberShadow would it be possible to install YUI on DAutoTest and set MINIFY=1?
(I picked YUI because it was the only compressor that is available in Debian stable.)

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @wilzbach!

Bugzilla references

Auto-close Bugzilla Severity Description
15449 enhancement CSS and JS files should be minified and concatenated

@CyberShadow
Copy link
Member

What do you think about doing the same thing as DFeed? https://github.com/CyberShadow/DFeed/blob/master/GNUmakefile#L4

It would give us control over the exact version of the tool. It seems likely that the output will change between versions, so that will help us avoid unexpected changes.

posix.mak Outdated
@@ -74,7 +74,7 @@
#
# DIFFABLE=1 Removes inclusion of all dynamic content and timestamps
# RELEASE=1 Release build (needs to be set for the `release` target)
# CSS_MINIFY=1 Minify the CSS via an online service
# MINIFY=1 Minify the CSS and JS (requires)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requires what?

@wilzbach
Copy link
Member Author

It seems likely that the output will change between versions, so that will help us avoid unexpected changes.

The last update has been in 2013 ...
But anyhow I like the idea and the less dependency on the host, the better :)

@wilzbach wilzbach force-pushed the yui-compressor branch 3 times, most recently from d38ac47 to 8fca0e1 Compare March 30, 2018 07:58
@wilzbach
Copy link
Member Author

BTW while I was at it, I added the HTML_COMPRESSOR too.

Before:

total 1,7M
drwxr-xr-x 10 seb users 4,0K Mar 30 09:59 ./
drwxr-xr-x 21 seb users 4,0K Mar 30 09:59 ../
-rw-r--r--  1 seb users 7,0K Mar 30 09:59 404.html
-rw-r--r--  1 seb users 7,8K Mar 30 09:59 acknowledgements.html
-rw-r--r--  1 seb users  23K Mar 30 09:59 areas-of-d-usage.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 articles/
-rw-r--r--  1 seb users 7,3K Mar 30 09:59 ascii-table.html
-rw-r--r--  1 seb users  15K Mar 30 09:59 bugstats.html
-rw-r--r--  1 seb users 6,9K Mar 30 09:59 calendar.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 changelog/
-rw-r--r--  1 seb users 7,6K Mar 30 09:59 community.html
-rw-r--r--  1 seb users  13K Mar 30 09:59 comparison.html
-rw-r--r--  1 seb users  17K Mar 30 09:59 concepts.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 css/
-rw-r--r--  1 seb users  11K Mar 30 09:59 D1toD2.html
-rw-r--r--  1 seb users  51K Mar 30 09:59 deprecate.html
-rw-r--r--  1 seb users  27K Mar 30 09:59 d-keyring.gpg
-rw-r--r--  1 seb users  566 Mar 30 09:59 d-keyring.gpg.sig
-rw-r--r--  1 seb users  48K Mar 30 09:59 dmd-freebsd.html
-rw-r--r--  1 seb users 7,4K Mar 30 09:59 dmd.html
-rw-r--r--  1 seb users  48K Mar 30 09:59 dmd-linux.html
-rw-r--r--  1 seb users  49K Mar 30 09:59 dmd-osx.html
-rw-r--r--  1 seb users  53K Mar 30 09:59 dmd-windows.html
drwxr-xr-x  3 seb users 4,0K Mar 30 09:59 docs/
-rw-r--r--  1 seb users 7,0K Mar 30 09:59 documentation.html
-rw-r--r--  1 seb users  26K Mar 30 09:59 download.html
-rw-r--r--  1 seb users 734K Mar 30 09:59 .dpl_rewrite_map.txt
-rw-r--r--  1 seb users  29K Mar 30 09:59 dstyle.html
-rw-r--r--  1 seb users  34K Mar 30 09:59 favicon.ico
-rw-r--r--  1 seb users  716 Mar 30 09:59 fetch-issue-cnt.php
-rw-r--r--  1 seb users 6,8K Mar 30 09:59 forum-template.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 foundation/
-rw-r--r--  1 seb users  21K Mar 30 09:59 glossary.html
-rw-r--r--  1 seb users 8,4K Mar 30 09:59 gpg_keys.html
-rw-r--r--  1 seb users 9,3K Mar 30 09:59 gsoc2011.html
-rw-r--r--  1 seb users 9,2K Mar 30 09:59 gsoc2012.html
-rw-r--r--  1 seb users 8,6K Mar 30 09:59 gsoc2012-template.html
-rw-r--r--  1 seb users  11K Mar 30 09:59 howto-promote.html
-rw-r--r--  1 seb users 4,3K Mar 30 09:59 .htaccess
-rw-r--r--  1 seb users  16K Mar 30 09:59 htod.html
drwxr-xr-x  4 seb users 4,0K Mar 30 09:59 images/
-rw-r--r--  1 seb users  47K Mar 30 09:59 index.html
-rw-r--r--  1 seb users  18K Mar 30 09:59 install.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 js/
-rw-r--r--  1 seb users  11K Mar 30 09:59 menu.html
-rw-r--r--  1 seb users  29K Mar 30 09:59 orgs-using-d.html
-rw-r--r--  1 seb users  49K Mar 30 09:59 overview.html
-rw-r--r--  1 seb users  12K Mar 30 09:59 rdmd.html
-rw-r--r--  1 seb users 7,4K Mar 30 09:59 resources.html
-rw-r--r--  1 seb users  100 Mar 30 09:59 robots.txt
-rw-r--r--  1 seb users 7,6K Mar 30 09:59 search.html
-rw-r--r--  1 seb users  19K Mar 30 09:59 sitemap.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 spec/
-rw-r--r--  1 seb users 9,1K Mar 30 09:59 tuple.html
-rw-r--r--  1 seb users 9,0K Mar 30 09:59 wc.html
-rw-r--r--  1 seb users 8,3K Mar 30 09:59 windbg.html

After:

total 1,6M
drwxr-xr-x 10 seb users 4,0K Mar 30 10:11 ./
drwxr-xr-x 21 seb users 4,0K Mar 30 10:10 ../
-rw-r--r--  1 seb users 6,2K Mar 30 09:59 404.html
-rw-r--r--  1 seb users 6,7K Mar 30 09:59 acknowledgements.html
-rw-r--r--  1 seb users  20K Mar 30 09:59 areas-of-d-usage.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 articles/
-rw-r--r--  1 seb users 6,5K Mar 30 09:59 ascii-table.html
-rw-r--r--  1 seb users  14K Mar 30 09:59 bugstats.html
-rw-r--r--  1 seb users 6,1K Mar 30 09:59 calendar.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 changelog/
-rw-r--r--  1 seb users 6,7K Mar 30 09:59 community.html
-rw-r--r--  1 seb users  11K Mar 30 09:59 comparison.html
-rw-r--r--  1 seb users  15K Mar 30 09:59 concepts.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 css/
-rw-r--r--  1 seb users 9,3K Mar 30 09:59 D1toD2.html
-rw-r--r--  1 seb users  47K Mar 30 09:59 deprecate.html
-rw-r--r--  1 seb users  27K Mar 30 09:59 d-keyring.gpg
-rw-r--r--  1 seb users  566 Mar 30 09:59 d-keyring.gpg.sig
-rw-r--r--  1 seb users  42K Mar 30 09:59 dmd-freebsd.html
-rw-r--r--  1 seb users 6,6K Mar 30 09:59 dmd.html
-rw-r--r--  1 seb users  42K Mar 30 09:59 dmd-linux.html
-rw-r--r--  1 seb users  42K Mar 30 09:59 dmd-osx.html
-rw-r--r--  1 seb users  47K Mar 30 09:59 dmd-windows.html
drwxr-xr-x  3 seb users 4,0K Mar 30 09:59 docs/
-rw-r--r--  1 seb users 6,2K Mar 30 09:59 documentation.html
-rw-r--r--  1 seb users  23K Mar 30 09:59 download.html
-rw-r--r--  1 seb users 734K Mar 30 09:59 .dpl_rewrite_map.txt
-rw-r--r--  1 seb users  27K Mar 30 09:59 dstyle.html
-rw-r--r--  1 seb users  34K Mar 30 09:59 favicon.ico
-rw-r--r--  1 seb users  716 Mar 30 09:59 fetch-issue-cnt.php
-rw-r--r--  1 seb users 6,0K Mar 30 09:59 forum-template.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 foundation/
-rw-r--r--  1 seb users  19K Mar 30 09:59 glossary.html
-rw-r--r--  1 seb users 7,6K Mar 30 09:59 gpg_keys.html
-rw-r--r--  1 seb users 8,4K Mar 30 09:59 gsoc2011.html
-rw-r--r--  1 seb users 8,4K Mar 30 09:59 gsoc2012.html
-rw-r--r--  1 seb users 7,7K Mar 30 09:59 gsoc2012-template.html
-rw-r--r--  1 seb users 8,9K Mar 30 09:59 howto-promote.html
-rw-r--r--  1 seb users 4,3K Mar 30 09:59 .htaccess
-rw-r--r--  1 seb users  14K Mar 30 09:59 htod.html
drwxr-xr-x  4 seb users 4,0K Mar 30 09:59 images/
-rw-r--r--  1 seb users  40K Mar 30 09:59 index.html
-rw-r--r--  1 seb users  16K Mar 30 09:59 install.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 js/
-rw-r--r--  1 seb users 9,0K Mar 30 09:59 menu.html
-rw-r--r--  1 seb users  26K Mar 30 09:59 orgs-using-d.html
-rw-r--r--  1 seb users  43K Mar 30 09:59 overview.html
-rw-r--r--  1 seb users  11K Mar 30 09:59 rdmd.html
-rw-r--r--  1 seb users 6,6K Mar 30 09:59 resources.html
-rw-r--r--  1 seb users  100 Mar 30 09:59 robots.txt
-rw-r--r--  1 seb users 6,6K Mar 30 09:59 search.html
-rw-r--r--  1 seb users  19K Mar 30 10:11 sitemap.html
drwxr-xr-x  2 seb users 4,0K Mar 30 09:59 spec/
-rw-r--r--  1 seb users 8,1K Mar 30 09:59 tuple.html
-rw-r--r--  1 seb users 8,2K Mar 30 09:59 wc.html
-rw-r--r--  1 seb users 7,5K Mar 30 09:59 windbg.html

It's not much, but it's a consistent ~10% decrease for almost for "free".
However, I guess this will mess up the diff, so I guess either we use MINIFY only for the deployment or why have a special HTML_MINIFY option.
What do you think?

@CyberShadow
Copy link
Member

There is little benefit from minifying HTML. It just strips whitespace, which compresses well anyway. It also makes inspecting HTML harder on the live website.

Same with CSS, though we have more comments there. Only JS generally minifies well.

@wilzbach
Copy link
Member Author

There is little benefit from minifying HTML. It just strips whitespace, which compresses well anyway.

So I tried apack test.tgz $(find web -name "*.html") for all html files generated with the html target:

~/dlang/dlang.org on yui-compressor [u] ± ll html.before.tgz
-rw-r--r-- 1 seb users 1,8M Mar 30 12:50 test2.tgz

~/dlang/dlang.org on yui-compressor [u] ± ll test.yui.tgz
-rw-r--r-- 1 seb users 1,1M Mar 30 12:50 test.tgz

Now an individual, randomly picked file:

~/dlang/dlang.org on yui-compressor [u] ± ll web.2/orgs-using-d.html
-rw-r--r-- 1 seb users 29K Mar 30 12:49 web.2/orgs-using-d.html
~/dlang/dlang.org on yui-compressor [u] ± ll web/orgs-using-d.html
-rw-r--r-- 1 seb users 26K Mar 30 12:50 web/orgs-using-d.html

And now the same file compressed:

~/dlang/dlang.org on yui-compressor [u] ± ll web.2/orgs-using-d.html.gz
-rw-r--r-- 1 seb users 5,8K Mar 30 12:49 web.2/orgs-using-d.html.gz

~/dlang/dlang.org on yui-compressor [u] ± ll web/orgs-using-d.html.gz
-rw-r--r-- 1 seb users 5,4K Mar 30 12:50 web/orgs-using-d.html.gz

It's still 10% (even with compression)

It also makes inspecting HTML harder on the live website.

Good point, but the developer tools use an auto-formated AST anyways and we could add .map files if this is really a problem.

@adamdruppe
Copy link
Contributor

I don't like minifying anything. It makes the live site a black box - you can't hit "view source" of something you see there and get an idea of where to edit for a PR, and complicates the build process (so making that PR is more fragile...) for, like you showed there, at best about a 10% difference (heck your example is even less than that! It is like 400 bytes.). I can see where that's sometimes worthwhile... but I don't think we're in that category. I doubt the 10% change is going to bump us into the next hosting tier or something like that, and the site isn't huge so in real numbers, that 10% means a download time difference of... a quarter of a second... on dial up.

Totally negligible on today's internet. gzip makes a big difference and is virtually free. But minify is a tiny difference that doesn't justify its own cost.

@CyberShadow
Copy link
Member

-rw-r--r-- 1 seb users 1,8M Mar 30 12:50 test2.tgz 
-rw-r--r-- 1 seb users 1,1M Mar 30 12:50 test.tgz

That's a suspiciously large difference. Where is the 40% reduction coming from?

The filenames in the command line and output are different. Was there an experiment error?

@wilzbach
Copy link
Member Author

Yes, sorry - the updated numbers for this experiment:

> mediainfo *.tgz
General
Complete name                            : web.compressed.tgz
Format                                   : GZip
Format profile                           : deflate
File size                                : 1.04 MiB

General
Complete name                            : web.old.tgz
Format                                   : GZip
Format profile                           : deflate
File size                                : 1.09 MiB

@CyberShadow
Copy link
Member

Thanks. I agree with @adamdruppe then, I don't think we win enough to compress HTML. There are more performance metrics to consider than file size.

@wilzbach
Copy link
Member Author

I don't think we win enough to compress HTML.

Hmm, can we at least compress CSS + JS files?
I did this as a separate PR (#2341), s.t. in case we ever want to include HTML compression this PR diff is an easy-to-access reference.

@wilzbach wilzbach closed this Apr 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants