-
Notifications
You must be signed in to change notification settings - Fork 214
Update to webpack 4 #809
Update to webpack 4 #809
Conversation
I've updated the above instructions for trying out this branch to prevent leftover packages from causing problems. (If you see errors like |
Preliminary builds using current and updated neutrino-preset-mozilla-frontend-infra on taskcluster-web: Build time went from 154 seconds to 46 seconds. Incredible. I noticed that in my updated preset, since the chunking logic is gone, I can no longer do this: Does this mean that |
Thank you for trying it out - nice improvement in build time! :-) I think there's more caching going on by default now (
Yeah I'm pretty sure that should work. Re the automatically generated chunk filenames - since this PR sets As such, I was thinking of filing an issue against webpack asking for the default when |
Build summaries:
Takeaways:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whew, finally made it through everything!
Everything here looks good to me, with the caveat that this will need to be rebased due to the uglify/babel minification discussions, and the image minification middleware patch.
Looks great so far. We will see where webpack-chain v5 and other things take us.
* Update webpack to ^4.6.0 * Update webpack-chain to ^4.6.0 * Update webpack-dev-server to ^3.1.3 * Update optimize-css-assets-webpack-plugin to ^4.0.0
Since webpack 4 now sets a default entry-point of `./src`, causing the error message to change for this test. (It doesn't seem worth creating the file to confirm it works, since from the error message we can tell the test got far enough that Neutrino correctly didn't complain about an explicit entry-point not being set.)
Adds the `mode` and `optimizations` documentation from: neutrinojs/webpack-chain#51
Bases `mode` on `NODE_ENV`, falling back to `'development'` for anything but production, since otherwise minification (and more) would be enabled during testing. This is the same approach used in create-react-app's WIP webpack 4 PR. For the effects `mode` has, see: https://webpack.js.org/concepts/mode/ https://github.com/webpack/webpack/blob/v4.6.0/lib/WebpackOptionsDefaulter.js#L26-L294 https://github.com/webpack/webpack/blob/v4.6.0/lib/WebpackOptionsApply.js#L301-L342
Since it's now enabled by default when `mode` is `'production'`: https://github.com/webpack/webpack/blob/v4.6.0/lib/WebpackOptionsDefaulter.js#L213-L215 https://github.com/webpack/webpack/blob/v4.6.0/lib/WebpackOptionsApply.js#L317-L318
webpack 4 now has an intelligent chunking feature that replaces `CommonsChunkPlugin` and avoids the need to create the fake `vendor` entrypoint to separate out third-party dependencies: https://webpack.js.org/plugins/split-chunks-plugin/ In addition, several of the plugins previously used are automatically enabled when `mode` is development: https://github.com/webpack/webpack/blob/v4.6.0/lib/WebpackOptionsDefaulter.js#L250-L259 https://github.com/webpack/webpack/blob/v4.6.0/lib/WebpackOptionsApply.js#L325-L328 (Note this differs slightly from our previous implementation, but if this results in long-term caching issues, issues should be filed to change the defaults upstream, and only set here as a last resort). However since `splitChunks` creates an unknown number of chunks with unpredictable filenames, it's no longer possible to pass an exact chunks list to `html-webpack-plugin`'s `chunks` option. We also can't omit the `chunks` option entirely, since for multi-entrypoint builds that would result in all assets being included for every page. As such, until `html-webpack-plugin` natively supports `splitChunks`, we have to use `html-webpack-include-sibling-chunks` to dynamically generate the `chunks` list: https://github.com/fenivana/html-webpack-include-sibling-chunks-plugin With the switch to `splitChunks`, `@neutrinojs/chunk` becomes simple enough that it's not really worth keeping separate from the web preset, particularly given that it relies on the sibling chunks workaround to actually be useable anyway. Note that webpack enables `splitChunks` in development too (which is great for dev-prod parity, and fixes a bug I was hitting migrating an app to Neutrino 8), which is why our overrides are no longer inside the `NODE_ENV === 'production'` block.
`uglify-js` (used by webpack 3) doesn't support ES6, so previously the web preset used `babel-minify` instead, to avoid having to transpile the generated javascript down to ES5 before minification. However webpack 4 now uses the new `uglify-es`, which does support ES6, and is much faster than `babel-minify`. As such, the web preset now defaults to `uglify-es` (which webpack 4 enables by default when `mode` is production). In addition, `babel-minify-webpack-plugin` has been found to be buggy when combined with `webpack-sources` > `1.0.1`, which required hacky workarounds in the form of yarn `resolutions` overrides and a forked version of `babel-minify-webpack-plugin`. This means we're on an old version of `webpack-sources`, which is missing fixes and performance improvements. It's likely before this PR is merged (or else before Neutrino 9 is released) that `@neutrinojs/babel-minify` will be completely detached from the web preset (and the workarounds removed) - but for now I've left it in, for easier A/B testing of the two minifiers. For more context, see: #748 It's also looking like we may want to adjust `webpack-chain` in the future, so that `optimization.minimizer` behaves like `plugins` and can have named entries for easier overriding. (To allow setting both JS and CSS minifiers there at once - especially given webpack 5 may also include a CSS minifier there too.)
Previously users were encouraged to create a fake `vendor` entrypoint which worked in conjunction with `@neutrinojs/chunk` to separate out third-party dependencies to a vendor chunk. However with `splitChunks`, manually specifying this entrypoint will actually result in larger builds - so we must ensure users correctly update their configuration.
Since the former is deprecated for CSS extraction, and is not compatible with webpack 4. The new plugin has a much simpler API: https://github.com/webpack-contrib/mini-css-extract-plugin I've also chosen to remove `css-hot-loader` - since it's unnecessary when using `style-loader` in development. ie option (2) from #802. (Once `mini-css-extract-plugin` supports HMR we can stop using `style-loader` entirely.) NB: `mini-css-extract-plugin` doesn't support having multiple plugin instances, so now we only have one instance for both standard CSS and module CSS. This will need testing to ensure everything still works.
Disabling `optimization.splitChunks` prevents hangs when running `yarn test`, and disabling `optimization.runtimeChunk` (which is enabled by the web preset) fixes zero tests being run. Example hang: https://travis-ci.org/mozilla-neutrino/neutrino-dev/jobs/369061914#L833
There's still a lot left untested, but this improves coverage significantly in the meantime.
We're going to stick with webpack 4's default of uglify-es for now, since it's much faster than babel-minify. Between that and the new webpack `optimization.minimize` and `optimization.minimizer` options, it's simpler to just document how to override the defaults directly, rather than provide a separate preset per alternative minifier.
It was required to work around a source map bug when using newer `webpack-sources` with `babel-minify-webpack-plugin`: webpack-contrib/babel-minify-webpack-plugin#68 However now that we're using `uglify-es` instead, we can stop pinning the `webpack-sources` version and pick up some of the bug/perf fixes: https://github.com/webpack/webpack-sources/releases
Updated stats
Looks like numbers are comparable within margin of error. Looks like the Babel minify error is back with the sources unpinned:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code lgtm, but it looks like we may have webpack-sources back, at least for now.
packages/web/index.js
Outdated
const styleMinify = require('@neutrinojs/style-minify'); | ||
const loaderMerge = require('@neutrinojs/loader-merge'); | ||
const devServer = require('@neutrinojs/dev-server'); | ||
const { join } = require('path'); | ||
const { resolve } = require('url'); | ||
const merge = require('deepmerge'); | ||
const HtmlWebpackIncludeSiblingChunksPlugin = require('html-webpack-include-sibling-chunks-plugin') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing trailing semicolon. Thanks eslint!
Being able to remove that workaround for me was another of the benefits of dropping official babel-minify support - particularly since further webpack-sources performance improvements are coming (eg webpack/webpack-sources#23). Instead of holding everyone back on an old version just in case someone wants to use Also my hunch is that the only reason we're seeing the sourcemap errors with babel-minify is because of the exact I'm going to create a minimal testcase repository without Neutrino to give the babel-minify-webpack-plugin maintainers something more to go on (in retrospect doing this sooner would have increased the chance of webpack-contrib/babel-minify-webpack-plugin#68 being fixed). |
Or alternatively removing the |
So re-reading the babel-minify-webpack-plugin issue I see that it occurs for all source map variants apart from cheap (and cheap is pointless when minifying, since everything is on one line), so I guess that leaves telling people to just disable sourcemaps in production if using babel-minify. (Which seems reasonable; it's not our fault it's buggy) |
OK, I can agree with that. Let's just not support it for now with a warning. |
So they catch the `false` case now too.
Now that the default `NODE_ENV` entry has been removed, the plugin has nothing to do unless the user had passed `env` in the preset options.
Since webpack 4 now loads the plugin by default in development. Leftover from #809.
Notable changes:
webpack
,webpack-dev-server
andoptimize-css-assets-webpack-plugin
.@neutrinojs/chunk
in favour of the splitChunks feature.@neutrinojs/babel-minify
in favour of the faster uglify-es (the fakevendor
entrypoint should now be removed from people's.neutrinorc.js
to avoid a bigger build then necessary).ModuleConcatenationPlugin
usage.NODE_ENV
from@neutrinojs/env
.webpack-sources
to v1.0.1.minify
options.vendor
entrypoints.For more details see the individual commit messages (most of the intermediate commits will fail to run standalone, but it still seemed preferable to split them up for easier review).
See here for the webpack 4 changelog:
https://github.com/webpack/webpack/releases/tag/v4.0.0
I'd love to have any and all feedback - including:
uglify-es
vsbabel-minify
compares for your use-casesTo try this branch out locally with
create-project
:git clone https://github.com/mozilla-neutrino/neutrino-dev.git
cd neutrino-dev && git checkout webpack-4
rm -rf node_modules/ packages/*/node_modules
(works around what appears to be a yarn workspaces bug, where stray webpack 3.x directories were left behind, causing errors about missingjson-loader
)yarn && yarn link:all
yarn global bin
directory is onPATH
cd .. && create-project <new directory name>
and answer the promptscd <new directory name>
yarn {start, build, lint, test}
as you would normallyAlternatively to use with an existing Neutrino 8 project:
cd <path to existing Neutrino 8 project>
neutrino
and@neutrinojs/*
packages frompackage.json
, and if there are any direct dependencies onwebpack
,webpack-dev-server
,html-webpack-plugin
or similar (eg since you import them in.neutrinorc.js
for advanced customisation) then adjust their versions to the latest available.rm -rf yarn.lock node_modules
(to prevent leftover hoisted webpack 3 packages from causing hard to debug errors)yarn && yarn link neutrino @neutrinojs/... @neutrinojs/...
(replace@neutrinojs/...
with whichever presets were previously referenced inpackage.json
- no need to list every Neutrino preset, just the top level dependencies)yarn {start, build, lint, test}
as you would normallypackage.json
andyarn.lock
, thenrm -rf node_modules && yarn
yarn lerna exec yarn unlink
Useful resources for reviewing:
mode
docssplitChunks
docsLeft to do:
uglify-es
orbabel-minify
by default (I'm leaning towards the former, see Compare speed/size of babel-minify vs uglify-es #748).@neutrinojs/babel-minify
(presuming we pickuglify-es
) should be completely separated to reduce dependencies/complexity. (Especially since a future minimizer-webpack-plugin might mean any custom handling in the web preset has to be rewritten anyway.)Fixes #737.
Fixes #802.
Closes #748.
Closes #768.
Closes #769.
Closes #766.