Ensure file.hash is always computed from sha1(file.data). #10330
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With the introduction of lazy compilation in Meteor 1.8, calling
inputFile.addJavaScript({
...
hash: inputFile.getSourceHash(),
...
}, function () {
return compiler.processFilesForTarget(inputFile);
});
becomes problematic, since inputFile.getSourceHash() is usually different
from compiler.processFilesForTarget(inputFile).hash, because the latter is
computed from the compiled code, whereas the former is computed from the
source code.
For example, when we use file.hash to cache imported module identifiers in
ImportScanner#_findImportedModuleIdentifiers, we really need to be using
the hash of the compiled code, since a single source module can be
compiled in different ways. If we cache based on the source hash, there's
a risk of reusing the scanned imports from the web.browser version for the
web.browser.legacy version, which can lead to all sorts of problems that
are only apparent in legacy browsers.
The quick fix is easy enough: BabelCompiler can simply stop including a
hash in the eager options to inputFile.addJavaScript. This fix can be
published as a minor update to the babel-compiler and ecmascript packages.
The remaining changes in this commit add another layer of defense against
this problem, by ignoring any hash options provided by compiler plugins,
in favor of simply computing the hash from the compiled data buffer.
These additional changes will become available in the next release of
Meteor (likely 1.8.1).
benjamn
pushed a commit
that referenced
this pull request
Nov 15, 2018
benjamn
pushed a commit
that referenced
this pull request
Nov 20, 2018
Hashes have a number of overlapping but not entirely redundant or equivalent purposes within the build system. Hashes of source code are important because they can be computed before compilation and processing, and thus are useful as keys for caching that expensive work. Source hashes remain useful even after compilation, as a way of reflecting the contributions of source-code-sensitive assets like source maps. However, source hashes do not tell the whole story, and using them as cache keys can be risky if the work that's being cached depends on generated code rather than source code, as we recently discovered with the findImportedModuleIdentifiers function. The preliminary fix for that problem (#10330) was to cache findImportedModuleIdentifiers using a hash of the generated code rather than the source hash. PR #10330 swung a bit too far in the direction of ignoring source hashes and considering only hashes of generated code. For example, the URLs of source maps share the hash of the corresponding resource, but source maps can change (because of superficial changes in the source code) without changing the generated code of the resource. Ignoring the source hash when computing source map URLs resulted in stale source maps with incorrect line numbers. A better solution seems to be to propagate the source hash (along with any hashes of intermediate generated artifacts) all the way through bundling, so that the final hash of any static resource reflects all information that could/should change the behavior of that static resource, including its source map, which embeds the exact source code of all contributing files in the sourcesContent property. At every step of the way, we merge all the input hashes into a single hash, so we don't have to keep juggling multiple hashes, thankfully. Sub-Resource Integrity (SRI) hashes still need to be computed from just the final contents of a given asset, so that the browser can verify those contents without knowing anything about the Meteor build system, but that's handled separately.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With the introduction of lazy compilation in Meteor 1.8, calling
becomes problematic, since
inputFile.getSourceHash()is usually different fromcompiler.processFilesForTarget(inputFile).hash, because the latter is computed from the compiled code, whereas the former is computed from the source code.For example, when we use
file.hashto cache imported module identifiers inImportScanner#_findImportedModuleIdentifiers, we really need to be using the hash of the compiled code, since a single source module can be compiled in different ways. If we cache based on the source hash, there's a risk of reusing the scanned imports from theweb.browserversion for theweb.browser.legacyversion, which can lead to all sorts of problems that are only apparent in legacy browsers.The quick fix is easy enough:
BabelCompilercan simply stop including ahashin the eager options toinputFile.addJavaScript. This fix can be published as a minor update to thebabel-compilerandecmascriptpackages.The remaining changes in this commit add another layer of defense against this problem, by ignoring any hash options provided by compiler plugins, in favor of simply computing the hash from the compiled data buffer. These additional changes will become available in the next release of Meteor (likely 1.8.0.1).