Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: remove zopfli to speed up webpack #10288

Merged
merged 1 commit into from Mar 16, 2019

Conversation

Projects
None yet
3 participants
@nolanlawson
Copy link
Collaborator

commented Mar 16, 2019

This PR removes zopfli compression, speeding up webpack compilation in production by ~7.5x (6m36s -> 0m53s).

Analysis

I did some profiling of the command:

RAILS_ENV=production bundle exec rails assets:precompile

after some digging, I found I could debug webpack using:

RAILS_ENV=production NODE_ENV=production node --inspect-brk /home/nolan/workspace/mastodon/node_modules/.bin/webpack --config /home/nolan/workspace/mastodon/config/webpack/production.js

So I took a trace. I think the screenshot speaks for itself:

Screenshot from 2019-03-15 19-00-50

Benchmark

My benchmark is the following:

rm -fr tmp/
RAILS_ENV=production bundle exec rails assets:precompile

Then I use the time command to time it.

Without the fix, this takes 6m36.437s in real time on my Dell XPS 13. With the fix, it takes 0m52.958s (~7.5x improvement).

Explanation

Zopfli outputs a very compact gzip format, but it's slow. Removing it speeds things up considerably.

Even though nginx's default gzip output may be less efficient than zopfli, I don't think it's reasonable to force every instance admin to run this command which takes a very long time. Maybe some admins aren't even using gzip – maybe they're using Brotli. They can also configure the gzip options in nginx, or run zopfli themselves.

As a future improvement, I don't think generating .gz files is even strictly necessary. Given this config, nginx should serve gzipped content no matter what – it's just that we don't necessarily control the gzip output.

@Gargron Gargron merged commit 8da5b8e into master Mar 16, 2019

11 checks passed

ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: check-i18n Your tests passed on CircleCI!
Details
ci/circleci: install Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.4 Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.5 Your tests passed on CircleCI!
Details
ci/circleci: install-ruby2.6 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.4 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.5 Your tests passed on CircleCI!
Details
ci/circleci: test-ruby2.6 Your tests passed on CircleCI!
Details
ci/circleci: test-webui Your tests passed on CircleCI!
Details
codeclimate All good!
Details

@Gargron Gargron deleted the nolan/remove-zopfli branch Mar 16, 2019

sumito3478 added a commit to orphos/mastodon that referenced this pull request Mar 19, 2019

@angristan

This comment has been minimized.

Copy link
Contributor

commented Mar 30, 2019

Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.