Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendor chunkhash changes when app code changes #1315

Closed
kevinrenskers opened this issue Jul 30, 2015 · 183 comments
Closed

Vendor chunkhash changes when app code changes #1315

kevinrenskers opened this issue Jul 30, 2015 · 183 comments

Comments

@kevinrenskers
Copy link

@kevinrenskers kevinrenskers commented Jul 30, 2015

I'm trying to split my vendor code from my app code and got it all working. However, every time I change some code in my app and create a new build, the vendor file gets a new chunkhash and thus is downloaded by the clients again. The contents are exactly the same though.

Here is a fully working small example showing the problem: https://github.com/kevinrenskers/chunkhash-problem

And as you can see, the contents of these files are 100% identical even though the chunkhash keeps changing:

screen shot 2015-07-30 at 22 47 59

@alexhancock
Copy link

@alexhancock alexhancock commented Jul 31, 2015

I am also experiencing this issue

@sokra sokra added the bug label Jul 31, 2015
@sokra
Copy link
Member

@sokra sokra commented Jul 31, 2015

Looks like a conflict between the plugin and the lastest changes. You could try a previous webpack version (thay had a bug that [chunkhash] didn't change when the content change).

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Jul 31, 2015

Any idea which webpack version I should try?

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Jul 31, 2015

Seems that going back to 1.10.1 works. 1.10.2 and up break the chunkhash as described above.

@sokra
Copy link
Member

@sokra sokra commented Jul 31, 2015

It doesn't "break" it, it fixes it. The chunk hash should change when the hashes of the child chunks change... ;)

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Jul 31, 2015

But if the contents of the vendor file don't change then its hash shouldn't change right? How else are we supposed to have long term caching if the filename changes every time I make a change in my app code?

@sokra
Copy link
Member

@sokra sokra commented Jul 31, 2015

That's caused by the chunk-manifest-webpack-plugin. It move the hashes of the child chunks out of the entry chunk. I propably need to rewrite it...

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Jul 31, 2015

I don't really understand. Isn't the whole point of chunk-manifest-webpack-plugin to remove the child chunk hashes exactly so that the contents of the entry chunk don't change and thus its hash should stay identical? It does work like that with 1.10.1.

Yet, you're saying that 1.10.2 have actually fixed this rather then breaking it. So, is the chunk-manifest-webpack-plugin doing something wrong? Or webpack? Or in other words, how am I supposed to have a working setup where changing the app code does not change the vendor's file name? :) Like I said it seems to work with 1.10.1 but your comments confuse me..

Are you able to get such a setup working with the latest webpack, as I've tried in the example project (linked above)?

@Zagitta
Copy link

@Zagitta Zagitta commented Aug 1, 2015

I'm attempting the same setup as @kevinrenskers and naturally have run into the same issue. If we possibly can come up with a workaround I'd appreciate it 👍

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Aug 9, 2015

Even going back to 1.10.1 only fixes the problem some of the time. Other times I create a build, the vendor file still gets a new file name. A fix would be extremely welcome.

@lsanwick
Copy link

@lsanwick lsanwick commented Aug 13, 2015

We're seeing this issue as well, any consistent fix would be great.

@airwin
Copy link

@airwin airwin commented Aug 14, 2015

same problem. I remove the chunkHash and use my own hash now :(.

@lsanwick
Copy link

@lsanwick lsanwick commented Aug 14, 2015

If you're willing to share, what method are you using to produce a hash?

@airwin
Copy link

@airwin airwin commented Aug 18, 2015

@lsanwick I just add my release date after the script path in the html file (eg: lib.js?v=20150818). It's a little stupid and all of this is unnecessary if webpack's chunkHash works corretly. :(

@Foxandxss
Copy link

@Foxandxss Foxandxss commented Sep 3, 2015

Makes sense, @bebraw and I started to investigate different solutions for this and it is just a bug. We hope this can be fixed :)

@adrian-e
Copy link

@adrian-e adrian-e commented Sep 11, 2015

+1 for this issue !

@michael-wolfenden
Copy link

@michael-wolfenden michael-wolfenden commented Sep 12, 2015

👍 for this issue as well

@bitfrost
Copy link

@bitfrost bitfrost commented Sep 14, 2015

Adding the webpack-md5-hash plugin in addition to the chunk-manifest plugin allowed me to work around this bug and get my non-changing vendor bundle to stay the same.

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Sep 14, 2015

Great solution! Works like a charm.

@dubrowgn
Copy link

@dubrowgn dubrowgn commented Sep 24, 2015

This all seems a little broken to me. Shouldn't the hash be applied at the file level instead of the chunk level? If a chunk produces an app.js and an app.css, I don't care if the chunk changed. I care if app.js and/or app.css files changed, because this ultimately determines if the client needs a new copy.

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Sep 28, 2015

Sadly adding webpack-md5-hash doesn't work either, because in the vendor files some references to modules in the main file change yet the hash stays the same. This completely breaks the entire app.

@bitfrost
Copy link

@bitfrost bitfrost commented Sep 30, 2015

Hmm, not seeing this behavior in my toy example. are you using OccurenceOrderPlugin ?
My plugin list:
new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
minChunks: Infinity
}),
new webpack.optimize.DedupePlugin(),
new ManifestPlugin(),
new ChunkManifestPlugin({
filename: 'chunk-manifest.json',
manifestVariable: 'webpackManifest'
}),
new ChunkManifestPluginWriter(),
new WebpackMd5Hash(),
new ManifestPlugin(),
new webpack.optimize.OccurenceOrderPlugin(),
new webpack.optimize.UglifyJsPlugin({
compress: {
warnings: false
}})

Hack to get around watch not triggering :
// Force output chunk-manifest after each build.
// TODO: figure out bug chunk-manifest-webpack-plugin that makes it miss
// webpack watches
class ChunkManifestPluginWriter {
apply(compiler) {
compiler.plugin('emit', (compilation, compileCallback) => {
const stats = compilation.getStats().toJson();
const result = {};
stats.assets.filter((asset) => {
return asset.name.endsWith('.js') & (asset.chunks[0] > 0);
}).forEach((asset) => {
result[String(asset.chunks[0])] = asset.name;
});
compilation.assets['chunk-manifest.json'] = new RawSource(JSON.stringify(result));
compilation.mainTemplate.plugin('require', (_, chunk, hash, chunkIdVar) => {
return _.replace('"CHUNK_MANIFEST"',
'window["webpackManifest"][' + chunkIdVar + ']');
});
compileCallback();
});
}
}

@kevinrenskers
Copy link
Author

@kevinrenskers kevinrenskers commented Sep 30, 2015

I'm using webpack.optimize.CommonsChunkPlugin, chunk-manifest-webpack-plugin, webpack-md5-hash, webpack.optimize.DedupePlugin, webpack.optimize.OccurenceOrderPlugin and webpack.optimize.UglifyJsPlugin.

Not sure what your ChunkManifestPluginWriter and ManifestPlugin plugins are?

@dmitry
Copy link

@dmitry dmitry commented Sep 30, 2015

Looks like related issue: #1479

@at0g
Copy link

@at0g at0g commented Oct 5, 2015

👎 for webpack-md5-hash + '[chunkhash]'

@danny-andrews
Copy link

@danny-andrews danny-andrews commented Sep 15, 2017

@gabn88 I think your issue with the hashes changing when you add a new entry can be fixed by using NamedChunksPlugin. If that works, I'll add it to my solution. Also, we should add an example for solving this problem to in order to put this ticket to rest once and for all. :)

Also, what do you mean by "adding an entry"?

@danny-andrews
Copy link

@danny-andrews danny-andrews commented Sep 15, 2017

@zaguini Agreed, it is very confusing. Have you tried the solution I posted? It's the minimal config I needed to get it working.

@songawee
Copy link
Contributor

@songawee songawee commented Oct 10, 2017

For those interested in using Webpack records to maintain consistent file hashes between builds, I wrote a bit about that here - https://medium.com/@songawee/long-term-caching-using-webpack-records-9ed9737d96f2.

@earslap
Copy link

@earslap earslap commented Oct 29, 2017

The saga continues...

I followed the official documentation on caching and it (thankfully) works fine. Then I added UglifyJS for tree shaking and compression, the output files themselves changed but the hashes didn't. That was surprising to me, since it can lead to complications (you minify but clients don't receive the new files since hashes don't change.)

Found out that chunkhash with the recommended method does the hashing before any other optimisations (like the ones done by UglifyJS.)

Then I found this: #4659

The proposed webpack-plugin-hash-output plugin seems to work, it does the hashing on top of the processed output (as far as I can understand) and seems to solve the issue... For now.

It's plugins on top of plugins on top of plugins at this point and I've been burned by this so many times over the years. Just when you think it all works, you change one config and it all crashes and burns until you add yet another plugin. The process is so so brittle.

Maybe this is not the right place but after all these years I'm convinced that this won't get any better, and the design is fundamentally flawed for this type of thing. Are there any commonly used solutions for long term caching outside of webpack? Like using webpack just for the vanilla bundling, and then something else goes through the HTML and linked resources to hash them separately and change the references in files?

In any case, if webpack-plugin-hash-output is the recommended solution now, is it worth adding to the documentation? The current documentation is leading people down a dangerous path (i.e. if they add anything to the plugin chain that processes / changes the files, the hashes won't change and at best people will waste their time diagnosing the issue like I did.)

@jjinux
Copy link

@jjinux jjinux commented Oct 30, 2017

earslap, I hear what you're saying, but it just reminds me of (https://nolanlawson.com/2017/03/05/what-it-feels-like-to-be-an-open-source-maintainer/).

To answer your question, perhaps the best way to minimize the danger of getting burned by this type of thing, you should cache for a shorter amount of time. That way, if they accidentally cache the non-minimized version, it won't stay cached forever.

Sorry, I can't offer you more, but at least it's better than nothing. Patches are certainly welcome.

@earslap
Copy link

@earslap earslap commented Oct 30, 2017

Oh no, I wasn't trying to put a blame on anyone involved with this project, sorry if my frustration made it sound that way.

I still think this is a fundamental design issue with the way webpack is architected. Brightest minds working on this deceptively simple issue for 2.5+ years, coming up with various solutions but none sticks. I don't know much about the internals of webpack but I assume something in there makes this extremely tricky.

If you strip all my frustrations from my earlier post, the meat should have been this: The documented method of cache busting (which I think summarises a happy path gathered from the discussions in this thread the past 2.5 years) will not work if any other plugin modifies the files along the way due to a config change, the hashes won't change - and that is dangerous (worse than hashes changing when they shouldn't as this can actually break things.) If my understanding is correct, there is a proposed solution that tries to fix this (here: #4659). Is this recommended, and if so should that be added to the documentation?

@chen86860
Copy link

@chen86860 chen86860 commented Nov 13, 2017

Just named the vendor can solve this problem 😆

code:

···
new webpack.optimize.CommonsChunkPlugin({
      names: 'vendor',
      minChunks: Infinity,
      filename: 'vendor.bundle.js',
 })
···
@michaelBenin
Copy link

@michaelBenin michaelBenin commented Dec 13, 2017

Anyone else coming here needing a solution, just create your own md5 hash.

const crypto = require('crypto');
const vendorList = [/* list of vendor files */];
const vendorHash = crypto.createHash('md5').update(vendorList.join('')).digest('hex');

Use vendorHash in the vendor filename options:

filename: `vendor.${vendorHash}.js

@michaelBenin
Copy link

@michaelBenin michaelBenin commented Dec 15, 2017

Regarding my comment above - that won't work as the bundle is referenced in the vendor file.

@NeoVance
Copy link

@NeoVance NeoVance commented Jan 1, 2018

@earslap if your cache does not invalidate when the contents of a file changes than perhaps you should look into fixing that.

Personally, I don't understand the obsession with using hashes in file names. There are far more simple, and easy to use methods of versioning files, and cache busting. This really isn't even an issue in webpack's wheel house to solve for you.

I feel like this is not something the webpack maintainers should worry about. This is not really webpack's role to fill.

@michael-wolfenden
Copy link

@michael-wolfenden michael-wolfenden commented Jan 2, 2018

@NeoVance curious to hear what alternatives there are to file hashing

@michaelBenin
Copy link

@michaelBenin michaelBenin commented Jan 2, 2018

@michael-wolfenden prior to webpack hashes and manifest, we versioned static files using the git sha or build number, as well as setting Cache-Control: max-age=31536000

@michael-wolfenden
Copy link

@michael-wolfenden michael-wolfenden commented Jan 2, 2018

@NeoVance @michaelBenin the issue with using the git sha or version number is that if you doing continuous deployment and depoying multiple times a day, the vendor bundle version constantly changes with changes to the app bundle version thus invalidating it's cache. This is the problem that file hashes are intended to solve.

@dalimian
Copy link

@dalimian dalimian commented Jan 26, 2018

@earslap had the same issue, @sokra wonder if this issue is gonna be addressed with webpack4 since you are rewriting treeshaking. I hope wp4 gets long term caching right, we been burned with it 3 times now with webpack 2-3

I am willing to help with the work

@maksimsemenov
Copy link

@maksimsemenov maksimsemenov commented Feb 9, 2018

@michaelBenin can you please explain a little bit more why custom vendor hash will not work. I mean what will happen if bundle had changed, but vendor file still referencing the old bundle. Will it cause any errors?
I was thinking of using it because all other approaches end with different hashes after CI builds, even if vendors were not changed. So want to know what caveats are here.

@michaelBenin
Copy link

@michaelBenin michaelBenin commented Feb 9, 2018

@maksimsemenov ~

First off we didn't get this working as intended, this issue is still open.

Basically we found that the vendor file has a reference of the app.js and that caused an error breaking JS.

@malliapi
Copy link

@malliapi malliapi commented Apr 29, 2018

is this still happening for webpack 4?

@edvardchen
Copy link

@edvardchen edvardchen commented May 3, 2018

To a certain extent, webpack 4 improved this situation, at least reducing the frequency of vendor hash changes, but the problem still exists.
See full code here
First,npm run build, output
image
If you just modify src/client.js, the vendor file's hash will remain unchanged unless your modified part changes the count of split chunks.

Like, comment one async import statement in src/client.js:

import React from 'react';

import { render } from 'react-dom';

// import('./App');

Then npm run build would output
image
Why? Because the vendors~app file still has a chunkId.

// head line in vendor-app file. This chunk Id is 1.
(window["webpackJsonp"] = window["webpackJsonp"] || []).push([[1],{

Maybe we need a special way to treat initial chunk. Treat that they are already installed without checking ?

@evilebottnawi
Copy link
Member

@evilebottnawi evilebottnawi commented May 3, 2018

I close this issue due with a very large number of questions/posts with very difference problems (include deprecated webpack versions). It's also hard to understand exactly what the problem - in the configuration or it's a real bug. Also please read https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31 (some steps no needs because it is already fixed in webpack). Feel free create new issue with minimum reproducible test repo.

In short ways:

webpackConfig.output = {
    chunkFilename: '[name]-[contenthash].chunk.js',
    filename: '[name]-[contenthash].js'
};

webpackConfig.plugins = [
   // ------------------------------------
    // Long Term Caching
    // ------------------------------------
    // More information https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31
    new webpack.NamedChunksPlugin(chunk => {
      if (chunk.name) {
        return chunk.name;
      }

      // eslint-disable-next-line no-underscore-dangle
      return [...chunk._modules]
        .map(m =>
          path.relative(
            m.context,
            m.userRequest.substring(0, m.userRequest.lastIndexOf("."))
          )
        )
        .join("_");
    }),
    new webpack.HashedModuleIdsPlugin()
];

Webpack@5 will have better long term caching out of box. Thanks!

@webpack webpack locked as off topic and limited conversation to collaborators May 3, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.