Webpack DLL Plugin

[mikecx]

Has anyone been able to get the Webpack DLL Plugin to work with webpacker? I followed the guide here: http://engineering.invisionapp.com/post/optimizing-webpack/ and ended up with a DLL that compiles, Webpack building, but I couldn't get javascript include to work as it was building a separate manifest which means the javascript_pack_tag helper couldn't find it.

[Judahmeek]

Had any further luck figuring this out @mikecx?

[mikecx]

@Judahmeek Sadly not too much. Haven't spent too much more time trying to get it working but it would definitely be a nice to have for the bundle speed increases.

[p0wl]

I got it working, the key is to run the vendor build (with DllPlugin) first and have that one write the manifest.json (using the webpack-manifest-plugin) first. The second build (webpacker with DllReferencePlugin) needs then to pick up the content of the already existing manifest.json using the seed option.

I will post our configuration tomorrow. We use it in dev, test and production.

[p0wl]

Here is the configuration that we use.

config/webpack/vendor.js:

// Note: You must restart bin/webpack-watcher for changes to take effect
/* eslint global-require: 0 */

const { join, resolve } = require('path')
const webpack = require('webpack')
const ManifestPlugin = require('webpack-manifest-plugin')

const environment = require('./environment')
const { paths } = require('./configuration.js')

environment.plugins.set(
  'DLL',
  new webpack.DllPlugin({
    path: join(resolve(paths.output, paths.entry), 'vendor-dll-manifest.json'),
    name: '[name]',
    context: resolve(paths.output, paths.entry)
  })
)

const config = environment.toWebpackConfig()

// list of packages that will be served from vendor.js
config.entry = {
  vendor: [
    'classnames',
    'core-js',
    'lodash/flatten',
    // ... all packages that you want to remove from the entry bundle and ship it via vendor.js
    'redux',
    'redux-thunk',
  ]
}

// set library name to 'vendor'
config.output.library = '[name]'
config.stats = 'errors-only'

module.exports = config

config/webpack/environment.js

const { basename, join, resolve } = require('path')
const { existsSync } = require('fs')

const { environment } = require('@rails/webpacker')
const webpack = require('webpack')
const ManifestPlugin = require('webpack-manifest-plugin')

const { paths, publicPath } = require('./configuration.js')

const manifestName = join(resolve(paths.output, paths.entry), 'manifest.json')
const manifest = existsSync(manifestName) ? require(manifestName) : {}

const vendorManifestName = join(resolve(paths.output, paths.entry), 'vendor-dll-manifest.json')
const vendorManifest = existsSync(vendorManifestName) ? require(vendorManifestName) : {}

environment.plugins.set(
  'Manifest',
  new ManifestPlugin({ fileName: 'manifest.json', publicPath, writeToFileEmit: true, cache: manifest })
);

environment.plugins.set(
  'DLL',
  new webpack.DllReferencePlugin({
    context: resolve(paths.output, paths.entry),
    manifest: vendorManifest
  })
);

module.exports = environment

bin/webpack / bin/webpack-dev-server

add:

"#{NODE_MODULES_PATH}/.bin/webpack", "--progress", "--color", "--config", "config/webpack/vendor.js",

app/views/index.html.haml

be sure to load the vendor bundle first

    = javascript_pack_tag 'vendor'
    = javascript_pack_tag 'web_client'

running bin/webpack will result in a public/packs/manifest.json (which is picked up from webpacker) like

{
  "vendor.js": "/packs/vendor-dc001cc5d1289798a659.js",
  "web_client.js": "/packs/web_client-080286df8829595eea88.js",
  "vendor.js.map": "/packs/vendor-68f231280e5d468a447b.js.map",
  "web_client.js.map": "/packs/web_client-8d5d1fdc18ffced5e6ba.js.map"
}

[gauravtiwari]

@p0wl Thanks for sharing. If you have time, perhaps you could open up a PR to document the usage
please 👍

[eriknygren]

@p0wl thanks for sharing this, could you please elaborate on what configuration.js should include? I'm trying to get this working but I'm having some issues getting the paths right. Cheers.

[p0wl]

configuration.js is pretty standard, no magic there:

// Common configuration for webpacker loaded from config/webpack/paths.yml

const { join, resolve } = require('path')
const { env } = require('process')
const { safeLoad } = require('js-yaml')
const { readFileSync } = require('fs')

const configPath = resolve('config', 'webpack')
const loadersDir = join(__dirname, 'loaders')
const paths = safeLoad(readFileSync(join(configPath, 'paths.yml'), 'utf8'))
const publicPath = `/${paths.entry}/`

module.exports = {
  env,
  paths,
  loadersDir,
  publicPath
}

and paths.yml as well:

# Restart webpack-watcher or webpack-dev-server if you make changes here
config: config/webpack
entry: packs
output: public
node_modules: node_modules
source: app/javascript
extensions:
  - .js
  - .jsx

What is not working for you? Maybe I can help when you post the errors that you get =)

[eriknygren]

Thanks for sharing that @p0wl, I got my DLL pack compiling now. My problem now is that I'm not having any success executing the DLL build and then the dev server.

First I had

# bin/webpack-dev-server
 cmd = [
  "#{NODE_MODULES_PATH}/.bin/webpack", "--progress", "--color", "--config", "config/webpack/vendor.js",
  "#{NODE_MODULES_PATH}/.bin/webpack-dev-server", "--progress", "--color",
  "--config", WEBPACK_CONFIG,
  "--host", LISTEN_HOST_ADDR,
  "--public", "#{HOSTNAME}:#{PORT}",
  "--port", PORT.to_s
] + ARGV

Which it didn't seem to like at all.

Then I tried:

# bin/webpack-dev-server
dll_cmd = [
  "#{NODE_MODULES_PATH}/.bin/webpack", "--progress", "--color", "--config", "config/webpack/vendor.js",
] + ARGV

cmd = [
  "#{NODE_MODULES_PATH}/.bin/webpack-dev-server", "--progress", "--color",
  "--config", WEBPACK_CONFIG,
  "--host", LISTEN_HOST_ADDR,
  "--public", "#{HOSTNAME}:#{PORT}",
  "--port", PORT.to_s
] + ARGV

Dir.chdir(APP_PATH) do
  p "execing the dll"
  exec env, *dll_cmd
  p "execing the dev server"
  exec env, *cmd
end

I was thinking they need to be in separate commands, and this got me halfway there, but it never reaches the "execing the dev server"part now, looks like it just exits as soon as it runs the first command. So would be great if you could elaborate a bit more on your bin/webpack* files setup for us uninitiated 🙏

[TheRusskiy]

@eriknygren did you get it working?

[TheRusskiy]

@p0wl what version of webpacker is it for?
also, for me <script src="/packs/vendor-b6ff2e635b8d0a001b39.js"></script> gives 404 (it does exist), I figure that's because it's on disk, and not in memory? How were you able to get around that?

[TheRusskiy]

Made it work with autodll-webpack-plugin.
Set up turned out to be more minimalistic, though a bit hack-ish and only got it working for dev so far (probably wouldn't want it on production anyway).
my webpack/development.js:

const environment = require("./environment")
const { join, resolve } = require('path')
const AutoDllPlugin = require('autodll-webpack-plugin');
environment.plugins.append("AutoDLL",
  new AutoDllPlugin({
    filename: '[name].dll.js',
    path: join(resolve("public", "packs"), 'vendor-dll'),
    debug: true,
    context: resolve("public", "packs"),
    entry: {
      vendor: [
        "actioncable",
        "...other packages to include"
      ]
    }
  })
)
module.exports = environment.toWebpackConfig()

then I had to also add to my layout this:

<% if Rails.env.development? %>
  <script type="text/javascript" charset="utf-8" src="/packs/vendor-dll/vendor.dll.js"></script>
<% end %>

[p0wl]

We are still at webpacker 3.0.2, didn't find the time to upgrade yet.

Actually, I think it should be used in production because that way your users won't have to re-download your vendor bundle after each deployment, because the extract vendor libraries won't change.

In a setup without the dll-plugin the chunkhash of your generated assets changes when you change anything in your frontend code. So the browser will download all changed assets again.
With the dll plugin you separate libraries from application code. The assets that contain the libraries will not change when you change application code and as it is extracted it will remain with the same chunkhash so the browser does not have to download the libraries again.

without dll plugin:
packs/app.js contains everything, application code, react, ... and will be quite huge

with dll plugin:
packs/app.<chunkhash>.js will only include application code and therefore very small
packs/vendor.<chunkhash>.js will include all common libraries (react, ...) and be huge, but will only change when you change the version of a library or add one.

if your vendor.dll.js is missing from packs/manifest.json you have to find a way that it stays in there.
For us the critical point was to set the cache parameter for the manifest-plugin:

const manifest = existsSync(manifestName) ? require(manifestName) : {}
new ManifestPlugin({ fileName: 'manifest.json', publicPath, writeToFileEmit: true, cache: manifest })

but it seems that the auto-dll-plugin is not able to work with the manifest plugin correctly: asfktz/autodll-webpack-plugin#58

So if you only want to use it for dev it is fine, but if you want to improve long term caching of your assets in production, you should try to get it working without the autodll-webpack-plugin.

[TheRusskiy]

@p0wl where I can I selectively import functions from packages, instead of importing the whole package. I may be wrong here, but to me it looked like DLL plugin just throws the whole package, so users are going to end up downloading stuff they don't need.
Also (and it's purely our use case) we may have several dashboards and each has it's own "pack" and their own libraries. Seems like a pain trying to configure dll plugin to make multiple caches for each dash, if we don't do it people are going to end up downloading packages from the whole other dashboard.