Tree shaking completely broken? #2867

Open
VladShcherbin opened this Issue Aug 13, 2016 · 30 comments
@VladShcherbin
VladShcherbin commented Aug 13, 2016 edited

updated with the latest versions
Webpack - 2.2.0
Babel es2015 preset - 6.18.0
OS X 10.12.2

Current

Tree shaking is not removing imports, not used in the app.

Expected

Tree shaking, removing not used imports.


The test case is a simple app with React & React Router v3. Link component is not used, but the whole router package is bundled.

import React from 'react'
import ReactDOM from 'react-dom'
import { Link } from 'react-router'

ReactDOM.render(
  <p>Hey</p>,
  document.getElementById('app')
)

Webpack & Rollup test repo

@TheLarkInn
Member

If you were to use this with babel-es2015-webpack does this still work. I think its worth verifying as ["es2015", {"loose": true, "modules": false}]] is a relatively new change to babel.

@VladShcherbin

@TheLarkInn yes, I tried both babel-preset-es2015-webpack and babel-preset-es2015-loose-native-modules presets.

Unfortunately, the result is the same.

@TheLarkInn
Member

What version did you jump from where it was working?

@VladShcherbin

@TheLarkInn I'm using it in a fresh project.

Got curious about the production file size after adding some packages and discovered this. I've seen previously issues and topics about the size, latest in create-react-app with no file size reduction.

I'll check another versions of webpack, maybe there will be difference.

@Kovensky
Collaborator

This seems to be a problem only in combination with reexports...

@VladShcherbin
VladShcherbin commented Aug 15, 2016 edited

Here is my small investigation. Everything was tested with the same config as above:

  • Webpack - 2.1.0-beta.20
  • Babel es2015 preset - 6.13.2
  • OS X 10.11

Simple react app:

import React from 'react'
import ReactDOM from 'react-dom'

ReactDOM.render(
  <p>Hey</p>,
  document.getElementById('app')
)

The production build is 146 kb.

Time for tests:

  1. Lets add some simple functions like this:
function fA() {
  console.log('I am a string')
}

We import some of them, but use only one:

import React from 'react'
import ReactDOM from 'react-dom'
import { fA, fB, fC, fD, fE, fF, fG } from './func'

fA()

ReactDOM.render(
  <p>Hey</p>,
  document.getElementById('app')
)

The size is still 146 kb as the function is pretty small. If we use all of them - the size is 150 kb.

Tree shaking is working.

  1. Lets make the same test with react stuff. Simple components:
import React from 'react'

const A = () => (
  <div>
    <p>Hey, I am a message</p>
  </div>
)

Lets use one component:

import React from 'react'
import ReactDOM from 'react-dom'
import { A, B, C, D, E, F, G } from './comp'

ReactDOM.render(
  <div>
    <A />
  </div>,
  document.getElementById('app')
)

The size is now 147 kb. With all components - 152 kb.

Tree shaking is working.

  1. Now we reexport the components. First, we create a file with named exports:
export { A, B, C, D, E, F, G } from './comp'

The same example as above with one component still gives us 147 kb.

Tree shaking is working.

  1. Same as previous, but with a *
export * from './comp'

Now the size with one component is 152 kb.

Tree shaking is not working (exports with *). Issue is here.

  1. Final test. We make two simple components and use them (as we know, tree shaking is working).
import React from 'react'
import ReactDOM from 'react-dom'
import { A } from './compA'
import { B } from './compB'

ReactDOM.render(
  <div>
    <A />
    <B />
  </div>,
  document.getElementById('app')
)

The size with one component - 147 kb, with both - 149 kb. Now, we take random library and import it in one of the components (lets take B):

import React from 'react'
import Select from 'react-select'

const B = () => (
  <div>
    <p>Hey, I am a message</p>
  </div>
)

We want to use only one component:

import React from 'react'
import ReactDOM from 'react-dom'
import { A } from './compA'
import { B } from './compB'

ReactDOM.render(
  <div>
    <A />
  </div>,
  document.getElementById('app')
)

Unfortunately, now the size is 185 kb.


So, the tree shaking actually removes the component, but some of the imports are included in the build (even if they were not used). Same thing happens with combined / reexported / nested files.

Any ideas why this happens with only some of the components and how to solve this?

@nylen
nylen commented Aug 16, 2016

I believe I'm having the same issue - tree shaking doesn't seem to be doing much of anything. The project is https://github.com/redmountainmakers/kilntroller-ui. I haven't narrowed it down to a specific bug but here is an example of a dependency I am definitely not using anywhere:

(image from the excellent [Webpack Visualizer](https://chrisbateman.github.io/webpack-visualizer/) tool)
@sokra
Member
sokra commented Aug 19, 2016

you could try it again with webpack beta 21, as I fixed an issue with reexports (exports * from '...')

@nylen
nylen commented Aug 20, 2016

No change in bundle size for the master branch of https://github.com/redmountainmakers/kilntroller-ui (444,561 bytes with both beta 20 and 21).

@nylen nylen added a commit to redmountainmakers/kilntroller-ui that referenced this issue Aug 20, 2016
@nylen nylen Upgrade to Webpack 2.1.0-beta.21 c257c6f
@blacksonic

@nylen @VladShcherbin have you tried turning on warnings for UglifyJS? it can tell a lot of useful information about skipped cleanup in unused classes and functions.

I have a very similar ticket with Typescript, where the generated class definitions considered as code with side effects.
#2899

@VladShcherbin

@sokra @blacksonic I actually have a very simple example repo here with webpack and rollup configs.

The index file imports Link from react-router, but never uses it. I expect it be removed from the production build, but it is still there (the whole react-router library I guess).

I'll give it a try with the new versions and warnings on and will post the result.

@blacksonic

Created an issue in UglifyJS, i think it is the source of the problem.
mishoo/UglifyJS2#1261

@VladShcherbin

Yep, I've tested the latest versions of both webpack/rollup - no changes with tree shaking. There are indeed uglify warnings, maybe this can help.

@blacksonic

There we go, lot of lines like this one

WARN: Side effects in initialization of unused variable replaceLocation [0:24087,4]

These variables remain in the minified version.

@nylen
nylen commented Aug 22, 2016 edited

I also have a ton of warnings from uglify (over 1000 of various kinds, including some about side effects). It looks like there's not an easy solution for these issues unfortunately. In https://github.com/redmountainmakers/kilntroller-ui/tree/try/closure-compiler I tried switching to closure-compiler instead - the build was a good bit smaller but I got a bunch of errors.

@drcmda
drcmda commented Sep 1, 2016 edited

Can someone explain in simple terms? I am trying to make sense of the informations present but i have no idea what's going on.

I am trying to get an outcome from three.js latest build, which finally features modules.

import * as THREE from 'three/src/Three.js';
let v = new THREE.Vector3();
console.log(v)

It seems to pretty much pull the entire lib, half a MB when it should be a few KB.

@modosc
modosc commented Sep 1, 2016 edited

i think webpack2 needs that to be:

import {Vector3} from "three/src/Three.js"
@Kovensky
Collaborator
Kovensky commented Sep 1, 2016

No, webpack 2 should be able to do tree shaking with namespace imports, as
long as the namespace export doesn't escape.

The important part is that the imported file must use ES6 exports.

On Fri, 2 Sep 2016 at 0:47 jonathan schatz notifications@github.com wrote:

i think webpack2 needs that to be:
'''
import {Vector3} from "three/src/Three.js"
'''


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#2867 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEdfaHY5722UN-TDX1OJCwnHQS4yFQqks5qlvOZgaJpZM4JjvHI
.

@arian
arian commented Sep 15, 2016 edited

I've hit the same problem:

index.js

import {sudoMakeMeASandwich} from './re-export';
console.log(sudoMakeMeASandwich); // make sure this one is used.

re-export.js

export {makeMeASandwich, sudoMakeMeASandwich} from './helpers';
export {unusedHelper} from './unused-helper';

helpers.js

export const makeMeASandwich = () => 'make sandwich: operation not permitted';
export const sudoMakeMeASandwich = () => 'one open faced club sandwich coming right up';

unused-helper.js

import * as unused2 from './unused-helper-2';
export const unusedHelper = unused2;

unused-helper-2.js

export const FOO = 'FOO';

Expected output

The expected output is a bundle with only the following modules: index, re-export and helpers. The unused modules should not be included, as those are only re-exported by re-export.js, but this export is never used.

Actual output

/******/ (function(modules) { // webpackBootstrap

/************************************************************************/
/******/ ([
/* 0 */
/***/ function(module, exports, __webpack_require__) {

"use strict";
/* harmony import */ var __WEBPACK_IMPORTED_MODULE_0__helpers__ = __webpack_require__(1);
/* harmony import */ var __WEBPACK_IMPORTED_MODULE_1__unused_helper__ = __webpack_require__(3);
/* unused harmony reexport makeMeASandwich */
/* harmony reexport (binding) */ __webpack_require__.d(exports, "a", function() { return __WEBPACK_IMPORTED_MODULE_0__helpers__["a"]; });
/* unused harmony reexport unusedHelper */



/***/ },
/* 1 */
/***/ function(module, exports, __webpack_require__) {

"use strict";
/* unused harmony export makeMeASandwich */
/* harmony export (binding) */ __webpack_require__.d(exports, "a", function() { return sudoMakeMeASandwich; });
var makeMeASandwich = function makeMeASandwich() {
  return 'make sandwich: operation not permitted';
};
var sudoMakeMeASandwich = function sudoMakeMeASandwich() {
  return 'one open faced club sandwich coming right up';
};

/***/ },
/* 2 */
/***/ function(module, exports, __webpack_require__) {

"use strict";
/* harmony export (binding) */ __webpack_require__.d(exports, "FOO", function() { return FOO; });
var FOO = 'FOO';

/***/ },
/* 3 */
/***/ function(module, exports, __webpack_require__) {

"use strict";
/* harmony import */ var __WEBPACK_IMPORTED_MODULE_0__unused_helper_2__ = __webpack_require__(2);
/* unused harmony export unusedHelper */

var unusedHelper = __WEBPACK_IMPORTED_MODULE_0__unused_helper_2__;

/***/ },
/* 4 */
/***/ function(module, exports, __webpack_require__) {

"use strict";
/* harmony import */ var __WEBPACK_IMPORTED_MODULE_0__re_export__ = __webpack_require__(0);

console.log(__webpack_require__.i(__WEBPACK_IMPORTED_MODULE_0__re_export__["a" /* sudoMakeMeASandwich */])());

/***/ }
/******/ ]);

Rollup

As a small comparison, rollup outputs only:

const sudoMakeMeASandwich = () => 'one open faced club sandwich coming right up';
console.log(sudoMakeMeASandwich());
@donaldpipowitch

FYI: I created another minimal test case for tree shaking with webpack 2.

For me tree shaking works as long as the return value of an import is not used inside another function - even if this function is never imported itself. E.g. if you take this lib logBaz can be tree shaked, because it is just re-exported. logFizz and logBuzz can be tree shaked, because they are a first level export. But: Sadly foo or doBar can never be tree shaked even though logFoo and logBar are never imported.

@andreigabreanu

FYI - although not sure if it's actually the same issue or not - but here is an example that doesn't work with Webpack but does work with rollup (you can try it on their website, that's how i tested it). Basically we have this structure:

// index.js
import { X } from "./tmp";
const x = new X(); // stop here => should bundle nothing
console.log(x); // stop here => should bundle only X 

// tmp.js
export {X} from "./X";
export {Y} from "./Y";

// X.js
export class X {
	methodFromX() {}
}

// Y.js
export class Y {
	methodFromY() {}
}

It bundles both files even though (using uglifyjs) in the end (search for "methodFromY")

@VladShcherbin
VladShcherbin commented Dec 19, 2016 edited

@andreigabreanu with such a simple example it may work, but with daily projects that use another libraries it does not.

Here is my simple test repo, super simple example with React & React Router. React Router was chosen as its creators tried to make it compatible with rollup/webpack tree-shaking, but it still does not work with the latest versions.

@oliviertassinari oliviertassinari referenced this issue in oliviertassinari/browser-metrics Dec 28, 2016
Open

Exports #1

@danbucholtz

Any update on this? I am surprised to see this not a part of the 2.x milestones. This is a major issue IMO because this is one of the "killer apps" of Webpack 2.x and it doesn't seem to be working as one would expect for even basic examples.

What can we do to help expedite this?

Thanks,
Dan

@w3apps
w3apps commented Jan 10, 2017

Tree shaking still not working as expected with rc.3

@Swizec
Swizec commented Jan 11, 2017

Adding another datapoint: Tree shaking seems ineffective in real-world projects.

I have a vendor.js chunk with a bunch of libraries (React, ReactDOM, MobX, etc.). Using Babel 6.14.0 and Webpack 2.2.0.rc.3 the file compiles to 2.34MB without minification.

Enabling minification brings it down to 939kB.

Disabling unused and dead_code flags on UglifyJsPlugin adds 30kB. This implies tree shaking was only able to remove 4% of the library code. That does not sound like a lot.

Adding/removing the {modules: false} option on Babel does nothing. Potentially (probably) the libs are precompiled to use CommonJS.

The initial large file size leads me to believe the libraries in question are not pre-minified.

Are my expectations of what tree shaking can do too high? Do we have to wait for libraries to stop distributing compiled versions?

@mstijak
mstijak commented Jan 11, 2017 edited

I did a small investigation a while ago. Based on a couple of issue threads and HackerNews comments, I think that proper tree-shaking (rollup way) will be implemented after v2 is released.
There is also an open issue in UglifyJS which is preventing good tree-shaking results right now. mishoo/UglifyJS2#1261

@jhnns
Member
jhnns commented Jan 11, 2017

Are my expectations of what tree shaking can do too high? Do we have to wait for libraries to stop distributing compiled versions?

Exactly. Most libraries still ship with CommonJS that's why it will take some time until tree-shaking will be actual effective.

@Akkuma
Contributor
Akkuma commented Jan 16, 2017

So I'm suffering from a similar problem. The issue is I have ES6 modules and within one of them it uses a CommonJS package. Webpack knows that I am not using the module that relies upon that cjs package, but it is still including the cjs package, but not on any of the ES6 module code that calls anything inside of the cjs package.

A simple pseudo code of it:

my-package
  reexport A
  reexport B
B
  import cjs-wrapper
cjs-wrapper
  require cjs-package

index
  import A from my-package

So I only use A, and cjs-package still appears in it, which clearly seems wrong.

@TheLarkInn
Member

Thank you everyone for keeping this issue up to date. I have to apologize for not communicating the update on how we plan to tackle this issue.

So after we have finished our 2.3 regression milestone for bugfixes for webpack 2(.2) final, we start on features.

Therefore I'm going to add this to our feature milestone.

So tl;dr mishoo/UglifyJS2#1261 is the biggest hurdle on solving this problem. So we either have to:

See if implement scope hoisting (rollup features) or build a new general purpose optimizer that supports a level of program flow analysis.

We would really appreciate as much brainstorming as possible to how we could solve this.

@danbucholtz
danbucholtz commented Jan 23, 2017 edited

Tree shaking should occur independently from minification/dead code removal IMO if possible. Otherwise it really doesn't rely on Webpack, it relies on a different 3rd party tool.

For example, when I run Rollup to produce a bundle, it's tree shaken without any sort of minification or dead code removal.

This is important for the Ionic Framework team because in order to support Webpack, Closure Compiler and Rollup independently, we run minification/dead code removal as a separate step independent from the bundling process.

Thanks,
Dan

@danbucholtz danbucholtz referenced this issue in driftyco/ionic-app-scripts Jan 23, 2017
Closed

slow launch time #670

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment