New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendor chunkhash changes when app code changes #1315

Closed
kevinrenskers opened this Issue Jul 30, 2015 · 183 comments

Comments

Projects
None yet
@kevinrenskers

kevinrenskers commented Jul 30, 2015

I'm trying to split my vendor code from my app code and got it all working. However, every time I change some code in my app and create a new build, the vendor file gets a new chunkhash and thus is downloaded by the clients again. The contents are exactly the same though.

Here is a fully working small example showing the problem: https://github.com/kevinrenskers/chunkhash-problem

And as you can see, the contents of these files are 100% identical even though the chunkhash keeps changing:

screen shot 2015-07-30 at 22 47 59

@alexhancock

This comment has been minimized.

Show comment
Hide comment
@alexhancock

alexhancock Jul 31, 2015

I am also experiencing this issue

alexhancock commented Jul 31, 2015

I am also experiencing this issue

@sokra sokra added the bug label Jul 31, 2015

@sokra

This comment has been minimized.

Show comment
Hide comment
@sokra

sokra Jul 31, 2015

Member

Looks like a conflict between the plugin and the lastest changes. You could try a previous webpack version (thay had a bug that [chunkhash] didn't change when the content change).

Member

sokra commented Jul 31, 2015

Looks like a conflict between the plugin and the lastest changes. You could try a previous webpack version (thay had a bug that [chunkhash] didn't change when the content change).

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Jul 31, 2015

Any idea which webpack version I should try?

kevinrenskers commented Jul 31, 2015

Any idea which webpack version I should try?

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Jul 31, 2015

Seems that going back to 1.10.1 works. 1.10.2 and up break the chunkhash as described above.

kevinrenskers commented Jul 31, 2015

Seems that going back to 1.10.1 works. 1.10.2 and up break the chunkhash as described above.

@sokra

This comment has been minimized.

Show comment
Hide comment
@sokra

sokra Jul 31, 2015

Member

It doesn't "break" it, it fixes it. The chunk hash should change when the hashes of the child chunks change... ;)

Member

sokra commented Jul 31, 2015

It doesn't "break" it, it fixes it. The chunk hash should change when the hashes of the child chunks change... ;)

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Jul 31, 2015

But if the contents of the vendor file don't change then its hash shouldn't change right? How else are we supposed to have long term caching if the filename changes every time I make a change in my app code?

kevinrenskers commented Jul 31, 2015

But if the contents of the vendor file don't change then its hash shouldn't change right? How else are we supposed to have long term caching if the filename changes every time I make a change in my app code?

@sokra

This comment has been minimized.

Show comment
Hide comment
@sokra

sokra Jul 31, 2015

Member

That's caused by the chunk-manifest-webpack-plugin. It move the hashes of the child chunks out of the entry chunk. I propably need to rewrite it...

Member

sokra commented Jul 31, 2015

That's caused by the chunk-manifest-webpack-plugin. It move the hashes of the child chunks out of the entry chunk. I propably need to rewrite it...

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Jul 31, 2015

I don't really understand. Isn't the whole point of chunk-manifest-webpack-plugin to remove the child chunk hashes exactly so that the contents of the entry chunk don't change and thus its hash should stay identical? It does work like that with 1.10.1.

Yet, you're saying that 1.10.2 have actually fixed this rather then breaking it. So, is the chunk-manifest-webpack-plugin doing something wrong? Or webpack? Or in other words, how am I supposed to have a working setup where changing the app code does not change the vendor's file name? :) Like I said it seems to work with 1.10.1 but your comments confuse me..

Are you able to get such a setup working with the latest webpack, as I've tried in the example project (linked above)?

kevinrenskers commented Jul 31, 2015

I don't really understand. Isn't the whole point of chunk-manifest-webpack-plugin to remove the child chunk hashes exactly so that the contents of the entry chunk don't change and thus its hash should stay identical? It does work like that with 1.10.1.

Yet, you're saying that 1.10.2 have actually fixed this rather then breaking it. So, is the chunk-manifest-webpack-plugin doing something wrong? Or webpack? Or in other words, how am I supposed to have a working setup where changing the app code does not change the vendor's file name? :) Like I said it seems to work with 1.10.1 but your comments confuse me..

Are you able to get such a setup working with the latest webpack, as I've tried in the example project (linked above)?

@Zagitta

This comment has been minimized.

Show comment
Hide comment
@Zagitta

Zagitta Aug 1, 2015

I'm attempting the same setup as @kevinrenskers and naturally have run into the same issue. If we possibly can come up with a workaround I'd appreciate it 👍

Zagitta commented Aug 1, 2015

I'm attempting the same setup as @kevinrenskers and naturally have run into the same issue. If we possibly can come up with a workaround I'd appreciate it 👍

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Aug 9, 2015

Even going back to 1.10.1 only fixes the problem some of the time. Other times I create a build, the vendor file still gets a new file name. A fix would be extremely welcome.

kevinrenskers commented Aug 9, 2015

Even going back to 1.10.1 only fixes the problem some of the time. Other times I create a build, the vendor file still gets a new file name. A fix would be extremely welcome.

@lsanwick

This comment has been minimized.

Show comment
Hide comment
@lsanwick

lsanwick Aug 13, 2015

We're seeing this issue as well, any consistent fix would be great.

lsanwick commented Aug 13, 2015

We're seeing this issue as well, any consistent fix would be great.

@airwin

This comment has been minimized.

Show comment
Hide comment
@airwin

airwin Aug 14, 2015

same problem. I remove the chunkHash and use my own hash now :(.

airwin commented Aug 14, 2015

same problem. I remove the chunkHash and use my own hash now :(.

@lsanwick

This comment has been minimized.

Show comment
Hide comment
@lsanwick

lsanwick Aug 14, 2015

If you're willing to share, what method are you using to produce a hash?

lsanwick commented Aug 14, 2015

If you're willing to share, what method are you using to produce a hash?

@airwin

This comment has been minimized.

Show comment
Hide comment
@airwin

airwin Aug 18, 2015

@lsanwick I just add my release date after the script path in the html file (eg: lib.js?v=20150818). It's a little stupid and all of this is unnecessary if webpack's chunkHash works corretly. :(

airwin commented Aug 18, 2015

@lsanwick I just add my release date after the script path in the html file (eg: lib.js?v=20150818). It's a little stupid and all of this is unnecessary if webpack's chunkHash works corretly. :(

@Foxandxss

This comment has been minimized.

Show comment
Hide comment
@Foxandxss

Foxandxss Sep 3, 2015

Makes sense, @bebraw and I started to investigate different solutions for this and it is just a bug. We hope this can be fixed :)

Foxandxss commented Sep 3, 2015

Makes sense, @bebraw and I started to investigate different solutions for this and it is just a bug. We hope this can be fixed :)

@adrian-e

This comment has been minimized.

Show comment
Hide comment
@adrian-e

adrian-e Sep 11, 2015

+1 for this issue !

adrian-e commented Sep 11, 2015

+1 for this issue !

@michael-wolfenden

This comment has been minimized.

Show comment
Hide comment
@michael-wolfenden

michael-wolfenden Sep 12, 2015

👍 for this issue as well

michael-wolfenden commented Sep 12, 2015

👍 for this issue as well

@bitfrost

This comment has been minimized.

Show comment
Hide comment
@bitfrost

bitfrost Sep 14, 2015

Adding the webpack-md5-hash plugin in addition to the chunk-manifest plugin allowed me to work around this bug and get my non-changing vendor bundle to stay the same.

bitfrost commented Sep 14, 2015

Adding the webpack-md5-hash plugin in addition to the chunk-manifest plugin allowed me to work around this bug and get my non-changing vendor bundle to stay the same.

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Sep 14, 2015

Great solution! Works like a charm.

kevinrenskers commented Sep 14, 2015

Great solution! Works like a charm.

@dubrowgn

This comment has been minimized.

Show comment
Hide comment
@dubrowgn

dubrowgn Sep 24, 2015

This all seems a little broken to me. Shouldn't the hash be applied at the file level instead of the chunk level? If a chunk produces an app.js and an app.css, I don't care if the chunk changed. I care if app.js and/or app.css files changed, because this ultimately determines if the client needs a new copy.

dubrowgn commented Sep 24, 2015

This all seems a little broken to me. Shouldn't the hash be applied at the file level instead of the chunk level? If a chunk produces an app.js and an app.css, I don't care if the chunk changed. I care if app.js and/or app.css files changed, because this ultimately determines if the client needs a new copy.

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Sep 28, 2015

Sadly adding webpack-md5-hash doesn't work either, because in the vendor files some references to modules in the main file change yet the hash stays the same. This completely breaks the entire app.

kevinrenskers commented Sep 28, 2015

Sadly adding webpack-md5-hash doesn't work either, because in the vendor files some references to modules in the main file change yet the hash stays the same. This completely breaks the entire app.

@bitfrost

This comment has been minimized.

Show comment
Hide comment
@bitfrost

bitfrost Sep 30, 2015

Hmm, not seeing this behavior in my toy example. are you using OccurenceOrderPlugin ?
My plugin list:
new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
minChunks: Infinity
}),
new webpack.optimize.DedupePlugin(),
new ManifestPlugin(),
new ChunkManifestPlugin({
filename: 'chunk-manifest.json',
manifestVariable: 'webpackManifest'
}),
new ChunkManifestPluginWriter(),
new WebpackMd5Hash(),
new ManifestPlugin(),
new webpack.optimize.OccurenceOrderPlugin(),
new webpack.optimize.UglifyJsPlugin({
compress: {
warnings: false
}})

Hack to get around watch not triggering :
// Force output chunk-manifest after each build.
// TODO: figure out bug chunk-manifest-webpack-plugin that makes it miss
// webpack watches
class ChunkManifestPluginWriter {
apply(compiler) {
compiler.plugin('emit', (compilation, compileCallback) => {
const stats = compilation.getStats().toJson();
const result = {};
stats.assets.filter((asset) => {
return asset.name.endsWith('.js') & (asset.chunks[0] > 0);
}).forEach((asset) => {
result[String(asset.chunks[0])] = asset.name;
});
compilation.assets['chunk-manifest.json'] = new RawSource(JSON.stringify(result));
compilation.mainTemplate.plugin('require', (_, chunk, hash, chunkIdVar) => {
return _.replace('"CHUNK_MANIFEST"',
'window["webpackManifest"][' + chunkIdVar + ']');
});
compileCallback();
});
}
}

bitfrost commented Sep 30, 2015

Hmm, not seeing this behavior in my toy example. are you using OccurenceOrderPlugin ?
My plugin list:
new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
minChunks: Infinity
}),
new webpack.optimize.DedupePlugin(),
new ManifestPlugin(),
new ChunkManifestPlugin({
filename: 'chunk-manifest.json',
manifestVariable: 'webpackManifest'
}),
new ChunkManifestPluginWriter(),
new WebpackMd5Hash(),
new ManifestPlugin(),
new webpack.optimize.OccurenceOrderPlugin(),
new webpack.optimize.UglifyJsPlugin({
compress: {
warnings: false
}})

Hack to get around watch not triggering :
// Force output chunk-manifest after each build.
// TODO: figure out bug chunk-manifest-webpack-plugin that makes it miss
// webpack watches
class ChunkManifestPluginWriter {
apply(compiler) {
compiler.plugin('emit', (compilation, compileCallback) => {
const stats = compilation.getStats().toJson();
const result = {};
stats.assets.filter((asset) => {
return asset.name.endsWith('.js') & (asset.chunks[0] > 0);
}).forEach((asset) => {
result[String(asset.chunks[0])] = asset.name;
});
compilation.assets['chunk-manifest.json'] = new RawSource(JSON.stringify(result));
compilation.mainTemplate.plugin('require', (_, chunk, hash, chunkIdVar) => {
return _.replace('"CHUNK_MANIFEST"',
'window["webpackManifest"][' + chunkIdVar + ']');
});
compileCallback();
});
}
}

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Sep 30, 2015

I'm using webpack.optimize.CommonsChunkPlugin, chunk-manifest-webpack-plugin, webpack-md5-hash, webpack.optimize.DedupePlugin, webpack.optimize.OccurenceOrderPlugin and webpack.optimize.UglifyJsPlugin.

Not sure what your ChunkManifestPluginWriter and ManifestPlugin plugins are?

kevinrenskers commented Sep 30, 2015

I'm using webpack.optimize.CommonsChunkPlugin, chunk-manifest-webpack-plugin, webpack-md5-hash, webpack.optimize.DedupePlugin, webpack.optimize.OccurenceOrderPlugin and webpack.optimize.UglifyJsPlugin.

Not sure what your ChunkManifestPluginWriter and ManifestPlugin plugins are?

@dmitry

This comment has been minimized.

Show comment
Hide comment
@dmitry

dmitry Sep 30, 2015

Looks like related issue: #1479

dmitry commented Sep 30, 2015

Looks like related issue: #1479

@at0g

This comment has been minimized.

Show comment
Hide comment
@at0g

at0g Oct 5, 2015

👎 for webpack-md5-hash + '[chunkhash]'

at0g commented Oct 5, 2015

👎 for webpack-md5-hash + '[chunkhash]'

@dmitry

This comment has been minimized.

Show comment
Hide comment
@dmitry

dmitry Oct 5, 2015

What about [hash], is it stays the same with webpack-md5-hash?

dmitry commented Oct 5, 2015

What about [hash], is it stays the same with webpack-md5-hash?

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Oct 8, 2015

I keep having this problem where in my vendor file a number like this changes (notice the highlighted 360, this used to be 359 in the build before this) yet the hash stays the same. No idea why that number changes in the first place.

screen shot 2015-10-08 at 16 04 33

kevinrenskers commented Oct 8, 2015

I keep having this problem where in my vendor file a number like this changes (notice the highlighted 360, this used to be 359 in the build before this) yet the hash stays the same. No idea why that number changes in the first place.

screen shot 2015-10-08 at 16 04 33

@kevinrenskers

This comment has been minimized.

Show comment
Hide comment
@kevinrenskers

kevinrenskers Oct 8, 2015

I think it's every time a new module is created and imported to my app. This then messes with the module numbers in vendor.js as well.

kevinrenskers commented Oct 8, 2015

I think it's every time a new module is created and imported to my app. This then messes with the module numbers in vendor.js as well.

@kimsagro1

This comment has been minimized.

Show comment
Hide comment
@kimsagro1

kimsagro1 Oct 8, 2015

I love webpack but it looks the hash story is completely broken (see the open issues below), making it impossible to implement long term caching.

  • Vendor chunkhash changes when app code changes #1315
  • Document what changes a hash/chunkhash #1486
  • Different hash on different computers #1479

This particular issue has been open for a couple of months now.

Is anyone actively looking into this issue?

kimsagro1 commented Oct 8, 2015

I love webpack but it looks the hash story is completely broken (see the open issues below), making it impossible to implement long term caching.

  • Vendor chunkhash changes when app code changes #1315
  • Document what changes a hash/chunkhash #1486
  • Different hash on different computers #1479

This particular issue has been open for a couple of months now.

Is anyone actively looking into this issue?

@sokra

This comment has been minimized.

Show comment
Hide comment
@sokra

sokra Oct 9, 2015

Member

I'll look into this after returning from vacation...

Member

sokra commented Oct 9, 2015

I'll look into this after returning from vacation...

@airwin

This comment has been minimized.

Show comment
Hide comment
@airwin

airwin Oct 9, 2015

👍 for webpack-md5-hash + '[chunkhash]'. It's really helpful.

airwin commented Oct 9, 2015

👍 for webpack-md5-hash + '[chunkhash]'. It's really helpful.

@bregenspan

This comment has been minimized.

Show comment
Hide comment
@bregenspan

bregenspan Oct 16, 2015

It seems like this is partly an issue of using integer-based module IDs. If there were a way to only refer to modules via fixed string IDs like suggested in #579 , that could be a good way around this?


(Edit): as stated by others, the webpack-md5-hash workaround appears to break in the case where:

  • there are two or more chunks loaded on the same page
  • module IDs change (e.g. due to addition/deletion of a module)
  • Chunk A includes the same modules it always did, but now they have different module IDs
  • webpack-md5-hash calculates the same hash for chunk A that it did before (because it's based on module contents, not the full rendered chunk with updated module IDs)
  • references to modules declared in Chunk A in another chunk stand the risk of pointing to older modules, because the old version of Chunk A with preexisting module IDs can still be delivered from cache as it has the same hash as it did before

Using "records" seems like one way around this, but also introduces some extra state that would be unnecessary were it possible to reference modules by path or some other stable identifier rather than numeric IDs.

bregenspan commented Oct 16, 2015

It seems like this is partly an issue of using integer-based module IDs. If there were a way to only refer to modules via fixed string IDs like suggested in #579 , that could be a good way around this?


(Edit): as stated by others, the webpack-md5-hash workaround appears to break in the case where:

  • there are two or more chunks loaded on the same page
  • module IDs change (e.g. due to addition/deletion of a module)
  • Chunk A includes the same modules it always did, but now they have different module IDs
  • webpack-md5-hash calculates the same hash for chunk A that it did before (because it's based on module contents, not the full rendered chunk with updated module IDs)
  • references to modules declared in Chunk A in another chunk stand the risk of pointing to older modules, because the old version of Chunk A with preexisting module IDs can still be delivered from cache as it has the same hash as it did before

Using "records" seems like one way around this, but also introduces some extra state that would be unnecessary were it possible to reference modules by path or some other stable identifier rather than numeric IDs.

@mpseidel

This comment has been minimized.

Show comment
Hide comment
@mpseidel

mpseidel Oct 26, 2015

+1 also ran into this problem today. It's weired, I am having 3 chunks:

  • app.bundle.js (chunkhash changes when app.bundle.js changes) <-ok
  • vendor1.bundle.js (chunkhash changes when app.bundle.js changes) <-problem
  • vendor2.bundle.js (chunkhash stays the same when app.bundle.js changes) <-ok

P.S.: using webpack-md5-hash seems to solve the problem for now in my case

mpseidel commented Oct 26, 2015

+1 also ran into this problem today. It's weired, I am having 3 chunks:

  • app.bundle.js (chunkhash changes when app.bundle.js changes) <-ok
  • vendor1.bundle.js (chunkhash changes when app.bundle.js changes) <-problem
  • vendor2.bundle.js (chunkhash stays the same when app.bundle.js changes) <-ok

P.S.: using webpack-md5-hash seems to solve the problem for now in my case

@bfin

This comment has been minimized.

Show comment
Hide comment
@bfin

bfin Oct 30, 2015

Would be great if we could use the same filename interpolation options (like <hashType>, <digestType>, <length>) that loader-utils makes available to file-loader.

bfin commented Oct 30, 2015

Would be great if we could use the same filename interpolation options (like <hashType>, <digestType>, <length>) that loader-utils makes available to file-loader.

@iliakan

This comment has been minimized.

Show comment
Hide comment
@iliakan

iliakan Nov 3, 2015

Apparently, chunk-manifest-plugin does extract the manifest, but the chunkhash is calculated prior to that.

iliakan commented Nov 3, 2015

Apparently, chunk-manifest-plugin does extract the manifest, but the chunkhash is calculated prior to that.

@gabn88

This comment has been minimized.

Show comment
Hide comment
@gabn88

gabn88 Mar 23, 2017

Ok, here's my final config on webpack 2 that seems to work (at least for me):

	  output: {
	      path: path.resolve('./assets/bundles/'),
	      filename: "[name]-[chunkhash].js",
	      chunkFilename: "[name]-[id]-[chunkhash].js"
	  },

No manifest in CommonsChunkPlugin (that seemed misuse of the CommonsChunkPlugin to me).

	        new ExtractTextPlugin({
	        	filename:'[name]-[chunkhash].css', 
	        }),
	        new webpack.NamedModulesPlugin(),
	    new ChunkManifestPlugin({
	      filename: "chunk-manifest.json",
	      manifestVariable: "webpackManifest"
	    })

Every file now gets a hash based on its content. This means that all extracted css files have a name based on their content, but also the different js files have a name based on their content.

The important part is: filename: "[name]-[chunkhash].js".

No need for Md5Hash of ChunkHash plugins.

EDIT: Ok, I switched to [contenthash] for the ExtractTextPlugin, just because it feels better. But I found a new issue which invalidates all hashes, which is when adding an entry. In that case webpackJsonp([19] , ... for example will become webpackJsonp([20], .... Which will invalidate the hash again 😕 I already use NamedModulePlugin, or for production webpack.HashedModuleIdsPlugin, but it does not seem to work. Any help would be appreciated!

gabn88 commented Mar 23, 2017

Ok, here's my final config on webpack 2 that seems to work (at least for me):

	  output: {
	      path: path.resolve('./assets/bundles/'),
	      filename: "[name]-[chunkhash].js",
	      chunkFilename: "[name]-[id]-[chunkhash].js"
	  },

No manifest in CommonsChunkPlugin (that seemed misuse of the CommonsChunkPlugin to me).

	        new ExtractTextPlugin({
	        	filename:'[name]-[chunkhash].css', 
	        }),
	        new webpack.NamedModulesPlugin(),
	    new ChunkManifestPlugin({
	      filename: "chunk-manifest.json",
	      manifestVariable: "webpackManifest"
	    })

Every file now gets a hash based on its content. This means that all extracted css files have a name based on their content, but also the different js files have a name based on their content.

The important part is: filename: "[name]-[chunkhash].js".

No need for Md5Hash of ChunkHash plugins.

EDIT: Ok, I switched to [contenthash] for the ExtractTextPlugin, just because it feels better. But I found a new issue which invalidates all hashes, which is when adding an entry. In that case webpackJsonp([19] , ... for example will become webpackJsonp([20], .... Which will invalidate the hash again 😕 I already use NamedModulePlugin, or for production webpack.HashedModuleIdsPlugin, but it does not seem to work. Any help would be appreciated!

@shasha-signifai

This comment has been minimized.

Show comment
Hide comment
@shasha-signifai

shasha-signifai Apr 11, 2017

I think the problem has been fixed in the 2.3.3 version. Please checkout this tutorial. It worked for me.
https://webpack.js.org/guides/code-splitting-libraries/

shasha-signifai commented Apr 11, 2017

I think the problem has been fixed in the 2.3.3 version. Please checkout this tutorial. It worked for me.
https://webpack.js.org/guides/code-splitting-libraries/

@antonholmquist

This comment has been minimized.

Show comment
Hide comment
@antonholmquist

antonholmquist Apr 16, 2017

I've struggled a lot with this the guide recommended by @shasha-signifai above actually seems to work for me: https://webpack.js.org/guides/code-splitting-libraries/

antonholmquist commented Apr 16, 2017

I've struggled a lot with this the guide recommended by @shasha-signifai above actually seems to work for me: https://webpack.js.org/guides/code-splitting-libraries/

@lomboboo

This comment has been minimized.

Show comment
Hide comment
@lomboboo

lomboboo Apr 28, 2017

I can confirm that it is working with solution that @danny-andrews showed above. Very important to understand that [hash] applies hash of a webpack build, so every file will have same hash, but [chunkhash] applies hash to every file separately.

So, this is how it works in my config:

output: {
  path: path.resolve( __dirname, "build" ),
  publicPath: '/',
  filename: "js/[name].[chunkhash].js"
},
plugins:[
...
new webpack.NamedModulesPlugin(),
new webpack.optimize.CommonsChunkPlugin( {
    name: [ "vendor", "manifest" ]
} ),
...
]

lomboboo commented Apr 28, 2017

I can confirm that it is working with solution that @danny-andrews showed above. Very important to understand that [hash] applies hash of a webpack build, so every file will have same hash, but [chunkhash] applies hash to every file separately.

So, this is how it works in my config:

output: {
  path: path.resolve( __dirname, "build" ),
  publicPath: '/',
  filename: "js/[name].[chunkhash].js"
},
plugins:[
...
new webpack.NamedModulesPlugin(),
new webpack.optimize.CommonsChunkPlugin( {
    name: [ "vendor", "manifest" ]
} ),
...
]
@tizmagik

This comment has been minimized.

Show comment
Hide comment
@tizmagik

tizmagik Apr 29, 2017

@lomboboo Apparently chunksSortMode isn't actually an option for CommonsChunkPlugin? https://github.com/webpack/webpack/blob/master/lib/optimize/CommonsChunkPlugin.js#L16-L24

tizmagik commented Apr 29, 2017

@lomboboo Apparently chunksSortMode isn't actually an option for CommonsChunkPlugin? https://github.com/webpack/webpack/blob/master/lib/optimize/CommonsChunkPlugin.js#L16-L24

@lomboboo

This comment has been minimized.

Show comment
Hide comment
@lomboboo

lomboboo May 1, 2017

@tizmagik you're right, just updated it. Anyway, it is working in my setup

lomboboo commented May 1, 2017

@tizmagik you're right, just updated it. Anyway, it is working in my setup

@timse

This comment has been minimized.

Show comment
Hide comment
@timse

timse May 21, 2017

Member

Given this ticket is still very active, I'll add this to the list of solutions:
https://medium.com/@schnibl/predictable-long-term-caching-with-webpack-d3eee1d3fa31

Member

timse commented May 21, 2017

Given this ticket is still very active, I'll add this to the list of solutions:
https://medium.com/@schnibl/predictable-long-term-caching-with-webpack-d3eee1d3fa31

@zaguiini

This comment has been minimized.

Show comment
Hide comment
@zaguiini

zaguiini Sep 15, 2017

It's still alive. I can't make it work. Tried everything. Confusing :(

zaguiini commented Sep 15, 2017

It's still alive. I can't make it work. Tried everything. Confusing :(

@danny-andrews

This comment has been minimized.

Show comment
Hide comment
@danny-andrews

danny-andrews Sep 15, 2017

@gabn88 I think your issue with the hashes changing when you add a new entry can be fixed by using NamedChunksPlugin. If that works, I'll add it to my solution. Also, we should add an example for solving this problem to in order to put this ticket to rest once and for all. :)

Also, what do you mean by "adding an entry"?

danny-andrews commented Sep 15, 2017

@gabn88 I think your issue with the hashes changing when you add a new entry can be fixed by using NamedChunksPlugin. If that works, I'll add it to my solution. Also, we should add an example for solving this problem to in order to put this ticket to rest once and for all. :)

Also, what do you mean by "adding an entry"?

@danny-andrews

This comment has been minimized.

Show comment
Hide comment
@danny-andrews

danny-andrews Sep 15, 2017

@zaguini Agreed, it is very confusing. Have you tried the solution I posted? It's the minimal config I needed to get it working.

danny-andrews commented Sep 15, 2017

@zaguini Agreed, it is very confusing. Have you tried the solution I posted? It's the minimal config I needed to get it working.

@songawee

This comment has been minimized.

Show comment
Hide comment
@songawee

songawee Oct 10, 2017

Contributor

For those interested in using Webpack records to maintain consistent file hashes between builds, I wrote a bit about that here - https://medium.com/@songawee/long-term-caching-using-webpack-records-9ed9737d96f2.

Contributor

songawee commented Oct 10, 2017

For those interested in using Webpack records to maintain consistent file hashes between builds, I wrote a bit about that here - https://medium.com/@songawee/long-term-caching-using-webpack-records-9ed9737d96f2.

@earslap

This comment has been minimized.

Show comment
Hide comment
@earslap

earslap Oct 29, 2017

The saga continues...

I followed the official documentation on caching and it (thankfully) works fine. Then I added UglifyJS for tree shaking and compression, the output files themselves changed but the hashes didn't. That was surprising to me, since it can lead to complications (you minify but clients don't receive the new files since hashes don't change.)

Found out that chunkhash with the recommended method does the hashing before any other optimisations (like the ones done by UglifyJS.)

Then I found this: #4659

The proposed webpack-plugin-hash-output plugin seems to work, it does the hashing on top of the processed output (as far as I can understand) and seems to solve the issue... For now.

It's plugins on top of plugins on top of plugins at this point and I've been burned by this so many times over the years. Just when you think it all works, you change one config and it all crashes and burns until you add yet another plugin. The process is so so brittle.

Maybe this is not the right place but after all these years I'm convinced that this won't get any better, and the design is fundamentally flawed for this type of thing. Are there any commonly used solutions for long term caching outside of webpack? Like using webpack just for the vanilla bundling, and then something else goes through the HTML and linked resources to hash them separately and change the references in files?

In any case, if webpack-plugin-hash-output is the recommended solution now, is it worth adding to the documentation? The current documentation is leading people down a dangerous path (i.e. if they add anything to the plugin chain that processes / changes the files, the hashes won't change and at best people will waste their time diagnosing the issue like I did.)

earslap commented Oct 29, 2017

The saga continues...

I followed the official documentation on caching and it (thankfully) works fine. Then I added UglifyJS for tree shaking and compression, the output files themselves changed but the hashes didn't. That was surprising to me, since it can lead to complications (you minify but clients don't receive the new files since hashes don't change.)

Found out that chunkhash with the recommended method does the hashing before any other optimisations (like the ones done by UglifyJS.)

Then I found this: #4659

The proposed webpack-plugin-hash-output plugin seems to work, it does the hashing on top of the processed output (as far as I can understand) and seems to solve the issue... For now.

It's plugins on top of plugins on top of plugins at this point and I've been burned by this so many times over the years. Just when you think it all works, you change one config and it all crashes and burns until you add yet another plugin. The process is so so brittle.

Maybe this is not the right place but after all these years I'm convinced that this won't get any better, and the design is fundamentally flawed for this type of thing. Are there any commonly used solutions for long term caching outside of webpack? Like using webpack just for the vanilla bundling, and then something else goes through the HTML and linked resources to hash them separately and change the references in files?

In any case, if webpack-plugin-hash-output is the recommended solution now, is it worth adding to the documentation? The current documentation is leading people down a dangerous path (i.e. if they add anything to the plugin chain that processes / changes the files, the hashes won't change and at best people will waste their time diagnosing the issue like I did.)

@jjinux

This comment has been minimized.

Show comment
Hide comment
@jjinux

jjinux Oct 30, 2017

earslap, I hear what you're saying, but it just reminds me of (https://nolanlawson.com/2017/03/05/what-it-feels-like-to-be-an-open-source-maintainer/).

To answer your question, perhaps the best way to minimize the danger of getting burned by this type of thing, you should cache for a shorter amount of time. That way, if they accidentally cache the non-minimized version, it won't stay cached forever.

Sorry, I can't offer you more, but at least it's better than nothing. Patches are certainly welcome.

jjinux commented Oct 30, 2017

earslap, I hear what you're saying, but it just reminds me of (https://nolanlawson.com/2017/03/05/what-it-feels-like-to-be-an-open-source-maintainer/).

To answer your question, perhaps the best way to minimize the danger of getting burned by this type of thing, you should cache for a shorter amount of time. That way, if they accidentally cache the non-minimized version, it won't stay cached forever.

Sorry, I can't offer you more, but at least it's better than nothing. Patches are certainly welcome.

@earslap

This comment has been minimized.

Show comment
Hide comment
@earslap

earslap Oct 30, 2017

Oh no, I wasn't trying to put a blame on anyone involved with this project, sorry if my frustration made it sound that way.

I still think this is a fundamental design issue with the way webpack is architected. Brightest minds working on this deceptively simple issue for 2.5+ years, coming up with various solutions but none sticks. I don't know much about the internals of webpack but I assume something in there makes this extremely tricky.

If you strip all my frustrations from my earlier post, the meat should have been this: The documented method of cache busting (which I think summarises a happy path gathered from the discussions in this thread the past 2.5 years) will not work if any other plugin modifies the files along the way due to a config change, the hashes won't change - and that is dangerous (worse than hashes changing when they shouldn't as this can actually break things.) If my understanding is correct, there is a proposed solution that tries to fix this (here: #4659). Is this recommended, and if so should that be added to the documentation?

earslap commented Oct 30, 2017

Oh no, I wasn't trying to put a blame on anyone involved with this project, sorry if my frustration made it sound that way.

I still think this is a fundamental design issue with the way webpack is architected. Brightest minds working on this deceptively simple issue for 2.5+ years, coming up with various solutions but none sticks. I don't know much about the internals of webpack but I assume something in there makes this extremely tricky.

If you strip all my frustrations from my earlier post, the meat should have been this: The documented method of cache busting (which I think summarises a happy path gathered from the discussions in this thread the past 2.5 years) will not work if any other plugin modifies the files along the way due to a config change, the hashes won't change - and that is dangerous (worse than hashes changing when they shouldn't as this can actually break things.) If my understanding is correct, there is a proposed solution that tries to fix this (here: #4659). Is this recommended, and if so should that be added to the documentation?

@chen86860

This comment has been minimized.

Show comment
Hide comment
@chen86860

chen86860 Nov 13, 2017

Just named the vendor can solve this problem 😆

code:

···
new webpack.optimize.CommonsChunkPlugin({
      names: 'vendor',
      minChunks: Infinity,
      filename: 'vendor.bundle.js',
 })
···

chen86860 commented Nov 13, 2017

Just named the vendor can solve this problem 😆

code:

···
new webpack.optimize.CommonsChunkPlugin({
      names: 'vendor',
      minChunks: Infinity,
      filename: 'vendor.bundle.js',
 })
···
@michaelBenin

This comment has been minimized.

Show comment
Hide comment
@michaelBenin

michaelBenin Dec 13, 2017

Anyone else coming here needing a solution, just create your own md5 hash.

const crypto = require('crypto');
const vendorList = [/* list of vendor files */];
const vendorHash = crypto.createHash('md5').update(vendorList.join('')).digest('hex');

Use vendorHash in the vendor filename options:

filename: `vendor.${vendorHash}.js

michaelBenin commented Dec 13, 2017

Anyone else coming here needing a solution, just create your own md5 hash.

const crypto = require('crypto');
const vendorList = [/* list of vendor files */];
const vendorHash = crypto.createHash('md5').update(vendorList.join('')).digest('hex');

Use vendorHash in the vendor filename options:

filename: `vendor.${vendorHash}.js

@michaelBenin

This comment has been minimized.

Show comment
Hide comment
@michaelBenin

michaelBenin Dec 15, 2017

Regarding my comment above - that won't work as the bundle is referenced in the vendor file.

michaelBenin commented Dec 15, 2017

Regarding my comment above - that won't work as the bundle is referenced in the vendor file.

@NeoVance

This comment has been minimized.

Show comment
Hide comment
@NeoVance

NeoVance Jan 1, 2018

@earslap if your cache does not invalidate when the contents of a file changes than perhaps you should look into fixing that.

Personally, I don't understand the obsession with using hashes in file names. There are far more simple, and easy to use methods of versioning files, and cache busting. This really isn't even an issue in webpack's wheel house to solve for you.

I feel like this is not something the webpack maintainers should worry about. This is not really webpack's role to fill.

NeoVance commented Jan 1, 2018

@earslap if your cache does not invalidate when the contents of a file changes than perhaps you should look into fixing that.

Personally, I don't understand the obsession with using hashes in file names. There are far more simple, and easy to use methods of versioning files, and cache busting. This really isn't even an issue in webpack's wheel house to solve for you.

I feel like this is not something the webpack maintainers should worry about. This is not really webpack's role to fill.

@michael-wolfenden

This comment has been minimized.

Show comment
Hide comment
@michael-wolfenden

michael-wolfenden Jan 2, 2018

@NeoVance curious to hear what alternatives there are to file hashing

michael-wolfenden commented Jan 2, 2018

@NeoVance curious to hear what alternatives there are to file hashing

@michaelBenin

This comment has been minimized.

Show comment
Hide comment
@michaelBenin

michaelBenin Jan 2, 2018

@michael-wolfenden prior to webpack hashes and manifest, we versioned static files using the git sha or build number, as well as setting Cache-Control: max-age=31536000

michaelBenin commented Jan 2, 2018

@michael-wolfenden prior to webpack hashes and manifest, we versioned static files using the git sha or build number, as well as setting Cache-Control: max-age=31536000

@michael-wolfenden

This comment has been minimized.

Show comment
Hide comment
@michael-wolfenden

michael-wolfenden Jan 2, 2018

@NeoVance @michaelBenin the issue with using the git sha or version number is that if you doing continuous deployment and depoying multiple times a day, the vendor bundle version constantly changes with changes to the app bundle version thus invalidating it's cache. This is the problem that file hashes are intended to solve.

michael-wolfenden commented Jan 2, 2018

@NeoVance @michaelBenin the issue with using the git sha or version number is that if you doing continuous deployment and depoying multiple times a day, the vendor bundle version constantly changes with changes to the app bundle version thus invalidating it's cache. This is the problem that file hashes are intended to solve.

@dalimian

This comment has been minimized.

Show comment
Hide comment
@dalimian

dalimian Jan 26, 2018

@earslap had the same issue, @sokra wonder if this issue is gonna be addressed with webpack4 since you are rewriting treeshaking. I hope wp4 gets long term caching right, we been burned with it 3 times now with webpack 2-3

I am willing to help with the work

dalimian commented Jan 26, 2018

@earslap had the same issue, @sokra wonder if this issue is gonna be addressed with webpack4 since you are rewriting treeshaking. I hope wp4 gets long term caching right, we been burned with it 3 times now with webpack 2-3

I am willing to help with the work

@maksimsemenov

This comment has been minimized.

Show comment
Hide comment
@maksimsemenov

maksimsemenov Feb 9, 2018

@michaelBenin can you please explain a little bit more why custom vendor hash will not work. I mean what will happen if bundle had changed, but vendor file still referencing the old bundle. Will it cause any errors?
I was thinking of using it because all other approaches end with different hashes after CI builds, even if vendors were not changed. So want to know what caveats are here.

maksimsemenov commented Feb 9, 2018

@michaelBenin can you please explain a little bit more why custom vendor hash will not work. I mean what will happen if bundle had changed, but vendor file still referencing the old bundle. Will it cause any errors?
I was thinking of using it because all other approaches end with different hashes after CI builds, even if vendors were not changed. So want to know what caveats are here.

@michaelBenin

This comment has been minimized.

Show comment
Hide comment
@michaelBenin

michaelBenin Feb 9, 2018

@maksimsemenov ~

First off we didn't get this working as intended, this issue is still open.

Basically we found that the vendor file has a reference of the app.js and that caused an error breaking JS.

michaelBenin commented Feb 9, 2018

@maksimsemenov ~

First off we didn't get this working as intended, this issue is still open.

Basically we found that the vendor file has a reference of the app.js and that caused an error breaking JS.

@malliapi

This comment has been minimized.

Show comment
Hide comment
@malliapi

malliapi Apr 29, 2018

is this still happening for webpack 4?

malliapi commented Apr 29, 2018

is this still happening for webpack 4?

@NoChapter

This comment has been minimized.

Show comment
Hide comment
@NoChapter

NoChapter May 3, 2018

To a certain extent, webpack 4 improved this situation, at least reducing the frequency of vendor hash changes, but the problem still exists.
See full code here
First,npm run build, output
image
If you just modify src/client.js, the vendor file's hash will remain unchanged unless your modified part changes the count of split chunks.

Like, comment one async import statement in src/client.js:

import React from 'react';

import { render } from 'react-dom';

// import('./App');

Then npm run build would output
image
Why? Because the vendors~app file still has a chunkId.

// head line in vendor-app file. This chunk Id is 1.
(window["webpackJsonp"] = window["webpackJsonp"] || []).push([[1],{

Maybe we need a special way to treat initial chunk. Treat that they are already installed without checking ?

NoChapter commented May 3, 2018

To a certain extent, webpack 4 improved this situation, at least reducing the frequency of vendor hash changes, but the problem still exists.
See full code here
First,npm run build, output
image
If you just modify src/client.js, the vendor file's hash will remain unchanged unless your modified part changes the count of split chunks.

Like, comment one async import statement in src/client.js:

import React from 'react';

import { render } from 'react-dom';

// import('./App');

Then npm run build would output
image
Why? Because the vendors~app file still has a chunkId.

// head line in vendor-app file. This chunk Id is 1.
(window["webpackJsonp"] = window["webpackJsonp"] || []).push([[1],{

Maybe we need a special way to treat initial chunk. Treat that they are already installed without checking ?

@evilebottnawi

This comment has been minimized.

Show comment
Hide comment
@evilebottnawi

evilebottnawi May 3, 2018

Member

I close this issue due with a very large number of questions/posts with very difference problems (include deprecated webpack versions). It's also hard to understand exactly what the problem - in the configuration or it's a real bug. Also please read https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31 (some steps no needs because it is already fixed in webpack). Feel free create new issue with minimum reproducible test repo.

In short ways:

webpackConfig.output = {
    chunkFilename: '[name]-[contenthash].chunk.js',
    filename: '[name]-[contenthash].js'
};

webpackConfig.plugins = [
   // ------------------------------------
    // Long Term Caching
    // ------------------------------------
    // More information https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31
    new webpack.NamedChunksPlugin(chunk => {
      if (chunk.name) {
        return chunk.name;
      }

      // eslint-disable-next-line no-underscore-dangle
      return [...chunk._modules]
        .map(m =>
          path.relative(
            m.context,
            m.userRequest.substring(0, m.userRequest.lastIndexOf("."))
          )
        )
        .join("_");
    }),
    new webpack.HashedModuleIdsPlugin()
];

Webpack@5 will have better long term caching out of box. Thanks!

Member

evilebottnawi commented May 3, 2018

I close this issue due with a very large number of questions/posts with very difference problems (include deprecated webpack versions). It's also hard to understand exactly what the problem - in the configuration or it's a real bug. Also please read https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31 (some steps no needs because it is already fixed in webpack). Feel free create new issue with minimum reproducible test repo.

In short ways:

webpackConfig.output = {
    chunkFilename: '[name]-[contenthash].chunk.js',
    filename: '[name]-[contenthash].js'
};

webpackConfig.plugins = [
   // ------------------------------------
    // Long Term Caching
    // ------------------------------------
    // More information https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31
    new webpack.NamedChunksPlugin(chunk => {
      if (chunk.name) {
        return chunk.name;
      }

      // eslint-disable-next-line no-underscore-dangle
      return [...chunk._modules]
        .map(m =>
          path.relative(
            m.context,
            m.userRequest.substring(0, m.userRequest.lastIndexOf("."))
          )
        )
        .join("_");
    }),
    new webpack.HashedModuleIdsPlugin()
];

Webpack@5 will have better long term caching out of box. Thanks!

@webpack webpack locked as off topic and limited conversation to collaborators May 3, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.