Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tree shaking #11164

Open
wants to merge 41 commits into
base: devel
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
51e2611
WIP dead code elimination
filipenevola Jun 23, 2020
d717389
Dead code elimination based on meteor build info, use babel instead o…
renanccastro Aug 17, 2020
be4cfbf
Fix build error
renanccastro Aug 17, 2020
cbadc54
Merge branch 'devel' into dead-code-elimination
renanccastro Aug 21, 2020
c310d9a
Support for custom global flags when minifying
renanccastro Aug 24, 2020
11fe73f
Support for custom global flags when running
renanccastro Aug 25, 2020
c65bd60
Prepare a map of imports to be used inside the bundler for removing u…
renanccastro Sep 4, 2020
d7e2ac8
Starting to remove some exports from the final bundle, there is still…
renanccastro Sep 8, 2020
90e39f0
Add api for setting sideEffects on meteor modules, start the changes …
renanccastro Sep 10, 2020
483029e
Using parent for deciding which files to include in the bundle in a c…
renanccastro Sep 11, 2020
f146f28
Don't stop analyzing files when first found, but accumulate the symbo…
renanccastro Sep 16, 2020
90ecfaa
Remove entires sub trees instead of only one node. Also store globall…
renanccastro Sep 23, 2020
9d6f99d
When adding the import tree to the files, do not run over the same no…
renanccastro Sep 24, 2020
0aa9d4a
Fix dynamic imports not working.
renanccastro Sep 24, 2020
4e719f6
Do not remove imports from node native modules if we are on the serve…
renanccastro Sep 29, 2020
cc6e08b
Remove log utils from tree shaking branch
renanccastro Sep 29, 2020
2bb3cf8
Remove log utils from tree shaking branch
renanccastro Sep 29, 2020
43c38d8
Remove log utils from tree shaking branch
renanccastro Sep 29, 2020
b712262
Implement better logic for getting side effects. If APP doesn't say i…
renanccastro Oct 6, 2020
7d45a7d
Don't tree shake import if is native module.
renanccastro Oct 6, 2020
58d6b49
Fix issue with dynamic imports not being included in the deps array.
renanccastro Oct 7, 2020
4291cdb
Import scanner fix for dynamic imports
renanccastro Oct 8, 2020
61bcc48
Use import status when deciding which tree to follow. Also remove sta…
renanccastro Oct 9, 2020
e8c2880
Fix issue with dynamic imports not being included in the deps array.
renanccastro Oct 14, 2020
eab13c8
Fix missing imports from importMap, reenable tree shaking.
renanccastro Oct 15, 2020
ed04c46
Fix removal of legit imports on dynamic info. Only remove imports of …
renanccastro Oct 15, 2020
e406055
Removing console log
renanccastro Oct 15, 2020
59aef3f
Fix missing imports - global variable is not always populated.
renanccastro Oct 16, 2020
70b2b02
Fix dynamic/static import status.
renanccastro Oct 20, 2020
2e5a998
Fix missing modules error.
renanccastro Oct 21, 2020
f374f15
Remove side effects free from base64. EJSON.newBinary is not a function.
renanccastro Oct 21, 2020
85e1295
Remove side effects free from base64. EJSON.newBinary is not a function.
renanccastro Oct 21, 2020
c327898
Get dead code elimination back
renanccastro Oct 22, 2020
3cd8664
Fix dead code elimination error
renanccastro Oct 22, 2020
4613a5a
Use buildMode for deciding if isTest or isProduction
renanccastro Oct 23, 2020
858a045
Use buildMode for deciding if isTest or isProduction
renanccastro Oct 23, 2020
059530e
Merge branch 'devel' into feature/tree-shaking
filipenevola Mar 31, 2021
1293c5d
Merge branch 'devel' into feature/tree-shaking
filipenevola Apr 3, 2021
cc19e51
Merge branch 'devel' into feature/tree-shaking
StorytellerCZ May 27, 2021
6875aef
Merge branch 'devel' into feature/tree-shaking
renanccastro Aug 23, 2021
a069789
Merge branch 'devel' into feature/tree-shaking
StorytellerCZ Sep 1, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
35 changes: 35 additions & 0 deletions packages/babel-compiler/babel.js
Expand Up @@ -47,5 +47,40 @@ Babel = {

getMinimumModernBrowserVersions: function () {
return Npm.require("@meteorjs/babel/modern-versions.js").get();
},
replaceMeteorInternalState: function(source, globalDefsMapping) {
try {
const globalDefsKeys = Object.keys(globalDefsMapping);
return Npm.require("@babel/core").transformSync(source, {
compact: false,
plugins: [
function replaceStateVars({types: t}) {
return {
visitor: {
MemberExpression: {
exit(path) {
const object = path.node.object.name;
const property = path.node.property.name;
const globalDefsForStart = object && globalDefsKeys.indexOf(object) > -1 && globalDefsMapping[object];
const mappingForEnds = property && globalDefsForStart
&& Object.keys(globalDefsForStart).indexOf(property) > -1
? globalDefsForStart[property] : null;

if (mappingForEnds !== null && path.parentPath.node.type !== "AssignmentExpression") {
path.replaceWith(
t.booleanLiteral(mappingForEnds === 'true' || mappingForEnds === true)
);
path.skip();
}
},
}
},
};
},
],
}).code;
} catch(e){
return source;
}
}
};
3 changes: 2 additions & 1 deletion packages/meteor/client_environment.js
Expand Up @@ -53,7 +53,8 @@ Meteor = {
* @static
* @type {Boolean}
*/
isModern: config.isModern
isModern: config.isModern,
...config.GLOBAL_DEFINITIONS
};

if (config.gitCommitHash) {
Expand Down
4 changes: 4 additions & 0 deletions packages/meteor/server_environment.js
Expand Up @@ -26,6 +26,9 @@ Meteor.settings = {};
if (process.env.METEOR_SETTINGS) {
try {
Meteor.settings = JSON.parse(process.env.METEOR_SETTINGS);
if(Meteor.settings.globalDefinitions){
Object.assign(Meteor, Meteor.settings.globalDefinitions);
}
} catch (e) {
throw new Error("METEOR_SETTINGS are not valid JSON.");
}
Expand All @@ -44,6 +47,7 @@ if (! Meteor.settings.public) {
// settings will be sent to the client.
if (config) {
config.PUBLIC_SETTINGS = Meteor.settings.public;
config.GLOBAL_DEFINITIONS = Meteor.settings.globalDefinitions ? Meteor.settings.globalDefinitions : {};
}

if (config && config.gitCommitHash) {
Expand Down
6 changes: 3 additions & 3 deletions packages/minifier-js/.npm/package/npm-shrinkwrap.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

63 changes: 52 additions & 11 deletions packages/minifier-js/minifier.js
@@ -1,35 +1,77 @@
var terser;

meteorJsMinify = function (source) {
const getGlobalDefsOptions = ({ arch, buildMode }) => ({
"Meteor.isServer": false,
"Meteor.isTest": buildMode === "test",
"Meteor.isDevelopment": false,
"Meteor.isClient": true,
"Meteor.isProduction": true,
"Meteor.isCordova": arch === 'web.cordova',
});

meteorJsMinify = function (source, options) {
var result = {};
var NODE_ENV = process.env.NODE_ENV || "development";

terser = terser || Npm.require("terser");
const globalDefs = getGlobalDefsOptions(options);

let customSettings = {};

/*
settings.json:
{
globalDefinitions: {
isAdmin: true
}
}
will replace Meteor.isAdmin with true
*/
if(process.env.METEOR_SETTINGS) {
const settings = JSON.parse(process.env.METEOR_SETTINGS);
customSettings = settings && settings.globalDefinitions || {};
}


var globalDefsMapping = Object.entries(globalDefs).reduce((acc, [from, to]) => {
const parts = from.split('.');
if (parts.length < 2) {
return acc;
}
const startValue = parts[0];
const endValue = parts[1];
return ({
...acc,
[startValue]: {
...acc[startValue], [endValue]: to
}
});
}, {});
globalDefsMapping.Meteor = {...globalDefsMapping.Meteor, ...customSettings};
try {
var terserResult = terser.minify(source, {
var optimizedCode = Babel.replaceMeteorInternalState(source, globalDefsMapping)
var terserResult = Meteor.wrapAsync(callback => terser.minify(optimizedCode, {
compress: {
drop_debugger: false,
unused: false,
unused: true,
dead_code: true,
global_defs: {
"process.env.NODE_ENV": NODE_ENV
}
"process.env.NODE_ENV": NODE_ENV,
"process.env.NODE_DEBUG": false,
},
// passes: 2
},
// Fix issue #9866, as explained in this comment:
// https://github.com/mishoo/UglifyJS2/issues/1753#issuecomment-324814782
// And fix terser issue #117: https://github.com/terser-js/terser/issues/117
safari10: true,
});

}).then((result) => callback(null, result)).catch((err) => callback(err, null)))();
if (typeof terserResult.code === "string") {
result.code = terserResult.code;
result.minifier = 'terser';
} else {
throw terserResult.error ||
new Error("unknown terser.minify failure");
new Error("unknown terser.minify failure");
}

} catch (e) {
// Although Babel.minify can handle a wider variety of ECMAScript
// 2015+ syntax, it is substantially slower than UglifyJS/terser, so
Expand All @@ -40,6 +82,5 @@ meteorJsMinify = function (source) {
result.code = Babel.minify(source, options).code;
result.minifier = 'babel-minify';
}

return result;
};
2 changes: 1 addition & 1 deletion packages/minifier-js/package.js
Expand Up @@ -4,7 +4,7 @@ Package.describe({
});

Npm.depends({
terser: "4.8.0"
terser: "5.3.2"
});

Package.onUse(function (api) {
Expand Down
1 change: 1 addition & 0 deletions packages/non-core/bundle-visualizer/package.js
Expand Up @@ -22,5 +22,6 @@ Package.onUse(function(api) {
'webapp',
]);
api.mainModule('server.js', 'server');
api.setSideEffects(true);
api.mainModule('client.js', 'client');
});
22 changes: 12 additions & 10 deletions packages/standard-minifier-js/plugin/minify-js.js
@@ -1,4 +1,4 @@
import { extractModuleSizesTree } from "./stats.js";
import {extractModuleSizesTree} from "./stats.js";

Plugin.registerMinifier({
extensions: ['js'],
Expand All @@ -8,9 +8,10 @@ Plugin.registerMinifier({
return minifier;
});

function MeteorBabelMinifier () {};
function MeteorBabelMinifier() {
};

MeteorBabelMinifier.prototype.processFilesForBundle = function(files, options) {
MeteorBabelMinifier.prototype.processFilesForBundle = function (files, options) {
var mode = options.minifyMode;

// don't minify anything for development
Expand Down Expand Up @@ -117,21 +118,22 @@ MeteorBabelMinifier.prototype.processFilesForBundle = function(files, options) {

files.forEach(file => {
// Don't reminify *.min.js.
if (/\.min\.js$/.test(file.getPathInBundle())) {
toBeAdded.data += file.getContentsAsString();
const content = file.getContentsAsString();
const filePath = file.getPathInBundle();
if (/\.min\.js$/.test(filePath)) {
toBeAdded.data += content;
} else {
var minified;

try {
minified = meteorJsMinify(file.getContentsAsString());

minified = meteorJsMinify(content, options);

if (!(minified && typeof minified.code === "string")) {
throw new Error();
}

} catch (err) {
var filePath = file.getPathInBundle();

maybeThrowMinifyErrorBySourceFile(err, file);

err.message += " while minifying " + filePath;
Expand All @@ -140,10 +142,10 @@ MeteorBabelMinifier.prototype.processFilesForBundle = function(files, options) {

const tree = extractModuleSizesTree(minified.code);
if (tree) {
toBeAdded.stats[file.getPathInBundle()] =
toBeAdded.stats[filePath] =
[Buffer.byteLength(minified.code), tree];
} else {
toBeAdded.stats[file.getPathInBundle()] =
toBeAdded.stats[filePath] =
Buffer.byteLength(minified.code);
}

Expand Down
10 changes: 5 additions & 5 deletions tools/cli/commands.js
Expand Up @@ -922,6 +922,7 @@ var buildCommands = {
architecture: { type: String },
"server-only": { type: Boolean },
'mobile-settings': { type: String },
'settings': { type: String },
server: { type: String },
"cordova-server-port": { type: String },
// Indicates whether these build is running headless, e.g. in a
Expand Down Expand Up @@ -1011,10 +1012,9 @@ var buildCommand = function (options) {
const serverOnly = options._bundleOnly || !!options['server-only'];

// options['mobile-settings'] is used to set the initial value of
// `Meteor.settings` on mobile apps. Pass it on to options.settings,
// which is used in this command.
if (options['mobile-settings']) {
options.settings = options['mobile-settings'];
// `Meteor.settings` on mobile apps.
if (options.settings) {
process.env.METEOR_SETTINGS = files.readFile(options.settings, 'utf8');
}

const appName = files.pathBasename(options.appDir);
Expand Down Expand Up @@ -1167,7 +1167,7 @@ ${Console.command("meteor build ../output")}`,
import { CordovaProject } from '../cordova/project.js';

cordovaProject = new CordovaProject(projectContext, {
settingsFile: options.settings,
settingsFile: options['mobile-settings'],
mobileServerUrl: utils.formatUrl(parsedMobileServerUrl),
cordovaServerPort: parsedCordovaServerPort });
if (buildmessage.jobHasMessages()) return;
Expand Down