Non-deterministic contenthash generation

[bschmeck]

We run a Rails app that is deployed to multiple webservers. During our deploy, we build packs on each machine from the same deployed code. We have very recently encountered an issue where this process has resulted in packs with different contenthash fingerprints in their filename. There are only 2 CSS files in the chunk, and it seems to be an issue of the order in which those modules are hashed when contenthash is being calculated.

• Operating System: CentOS 8
• Node Version: 10.19.0
• NPM Version: 6.13.4
• webpack Version: 4.41.5
• mini-css-extract-plugin Version: 0.8.2

Expected Behavior

Building packs for the same code on multiple machines should generate identical contenthash filenames.

Actual Behavior

The modules associated with a chunk are hashed in a non-deterministic way, resulting in different fingerprints on different machines.

Code

How Do We Reproduce?

I'm still working on a reproducable example, but the issue appears to be the order in which chunk.modulesIterable returns modules here:

mini-css-extract-plugin/src/index.js

Lines 227 to 231 in 1ffc393

	for (const m of chunk.modulesIterable) {
	if (m.type === MODULE_TYPE) {
	m.updateHash(hash);
	}
	}

modulesIterable is a SortableSet, but it appears that the code needs to explicitly call sort() on the set to retrieve values in order. Adding a call to chunk.modulesIterable.sort() before hashing the modules appears to be sufficient to generate a consistent fingerprint. If that seems to be a reasonable approach, I'm happy to open a PR but am struggling to put together a test for the change.

[alexander-akait]

We known about non-deterministic contenthash problems due source maps, please create reproducible test repo we need investigate, maybe you faced with a new problem

[yingzhe]

Left a comment here: #482
I think the problem is that when generating the contenthash generation is based on the order of modules inserted into chunk.modulesIterable, but when rendering content asset, the modules are first sorted by index2. If contenthash generation includes index2, maybe that would solve the issue?

[alexander-akait]

Solved for webpack@5 with and without source maps, for webpack@4 solved only without source maps (architecture problems), sorry for delay + webpack@5 generate real content hash in the production mode