Please make long term caching better supported out of the box

[fruchtose]

Do you want to request a feature or report a bug? Feature

What is the current behavior? Currently, predictable long term caching of output chunks is difficult to achieve. After reading this article, I was surprised to learn two things:

1. I have been doing predictable long-term caching wrong for a long time.
2. Every project is different, and the CommonsChunkPlugin integration for every project will be different. However, there are a number of concepts required for consistent long term caching which are not used by default, including the plugins NamedChunksPlugin & NamedModulesPlugin, and ensuring that all chunks have a module identifier.

The documentation for caching includes minimal explanation for using module identifiers instead of facilities for supporting long term caching:

To minimize the size of generated files, webpack uses identifiers instead of module names.

However, for production projects, this means that more work needs to be done to ensure that long term caching is possible since the defaults do not support it, and the rapid state of change in the webpack codebase (and supporting documentation) can easily be missed by devs on webpack projects.

What is the expected behavior? It would be nice to have sensible defaults that make long term caching easier, such as including NamedChunksPlugin and NamedModulesPlugin by default, and ensuring that all modules have an identifier. Additionally, in the future it would be nice to include config options that more simply expose some (pardon the pun) common use cases of CommonsChunkPluginbehaviors, like breaking out chunks into their own files, creating a runtime chunk, and including async children chunks in a parent chunk.

If this is a feature request, what is motivation or use case for changing the behavior? As a webpack user for over a year now, I still feel that the framework is very powerful—but that there are a number of default configurations that should be simpler to achieve. It is not clear that there is a worthy performance reason to not support long term caching by default, when it is not immediately clear that webpack requires additional configuration for long term caching in the first place.

[danny-andrews]

I second this. Getting long-term caching working involves a lot of trial-and-error, as evidenced by this issue. It also requires you to know too much about the internals of webpack in order to get it to work. Why should I have to know how webpack identifies modules internally in order to fingerprint my files deterministically? I think HashedModuleIdsPlugin should be used by default when process.env.NODE_ENV='production' (or at the very least included with the --optimize-minimize flag). Then the only thing you'd need to do to get caching working is to extract the runtime and use chunkhash in your output filename. E.g.

This:

output: {
  filename: '[name].[chunkhash].js'
},
plugins: [
  // Order matters. 'vendor' must come before 'runtime'
  new webpack.optimize.CommonsChunkPlugin({
    name: 'vendor'
  }),
  new webpack.optimize.CommonsChunkPlugin({
    name: 'runtime'
  })
]

instead of this:

output: {
  filename: '[name].[chunkhash].js'
},
plugins: [
  // Order matters. 'vendor' must come before 'runtime'
  new webpack.optimize.CommonsChunkPlugin({
    name: 'vendor'
  }),
  new webpack.optimize.CommonsChunkPlugin({
    name: 'runtime'
  }),
  new webpack.NamedModulesPlugin(),
  new webpack.NamedChunksPlugin()
]

[dmitriid]

All the various and extremely confusing combinations of multiple caching plugins should really be deprecated in favour of default behaviour at https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31

Moreover, this page should feature prominently in each and every page that discusses caching until caching is solved.

[danny-andrews-snap]

I got most of my example config from the official docs on caching. Not sure what's "extremely confusing" about it. Although, I do agree the one from the medium article is more robust.

[dmitriid]

@danny-andrews-snap

Thta page is just a "running start". If you need an actual caching solution, you have to follow the several-screens-long explanation in the medium post. However, caching as described in the post should work out of the box, and not require umpteen plugins with arcane configurations to get what you need in the first place.

On top of that, that entire documentation page is already outdated because CommonChunksPlugin is removed in webpack v4.

Thankfully, persistent caching is in the roadmap for v5.

[danny-andrews-snap]

Cool, I think it makes sense to point people to that article in the meantime.

[sokra]

webpack 5 has added a lot improvements for long term caching.
It now produces output files in a deterministic way, so that changes to on part of the application are very unlikely to affect another part of the application. Module and Chunk Ids use determinstic numeric ids (which is a good tradeoff between size, caching and not exposing internals). Exports are mangled in a determinstic way using 1-3 letter hashes depending on the number of exports. node_modules (once they reach a size threshold) are automatically split from application modules to respect the different change behavior.

All this should yield good caching behavior for caching via re-validation by default.

I can recommend to add the following options for even better long term caching:

output: {
  filename: "[contenthash].js",
},
optimization: {
  splitChunks: {
    chunks: "all"
  }
}

Currently we did not enable these options by default because they make it harder to use the output files generated by webpack. output.filename make the filename unguessable and splitChunks.chunks will generate multiple files that need to be included via script tag. To use output files with these settings you need to have some kind of process that will generate the HTML and script tags for you. To improve developer experience, especially for newcomers, this is not needed by default.

In future we might offer the ability to use an HTML file as entrypoint, which would allow to enable such optimizations by default.