New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance of webpack-dev-server with source maps #168

Closed
KidkArolis opened this Issue Feb 12, 2014 · 20 comments

Comments

Projects
None yet
6 participants
@KidkArolis

KidkArolis commented Feb 12, 2014

I'm working on this larg(ish) project where we currently use AMD. It works all right, we don't build the project in development and load each module individually. It takes about ~2.5 seconds to load the page with ~420 modules/templates.

I'm exploring switching to webpack (which so far looks amazing) for all the benefits that would add. I have our entire existing project building with webpack with no modifications to the project, which is awesome.

It takes 14s to build the project, but if I use webpack-dev-server, the rebuilds are impressively fast ~1s.

However, if I turn on devtool: "source-map", the rebuilds then take ~4s which is slower than loading all modules individually with Require.js where source maps are not needed.
I've tried using devtool: "eval" and that's fast, but it's not .. a perfect sourcemap (e.g. requires are all numbers now and pathInfo: true doesn't seem to work for me).

So, my question really is, why does sourcemap slowdown rebuilds so much and how could I try to go around it. E.g. Should I try to make more smaller chunks? Should I try to move some of the shared dependencies from the chunks into the entry bundle? Am I doing something wrong with having so many megabytes of JavaScript?

Here's my current output just to give you the idea of the size of the project.

Version: webpack 1.0.0-rc3
Time: 14180ms
      Asset     Size  Chunks             Chunk Names
  bundle.js   985492       0  [emitted]  main       
1.bundle.js   437111       1  [emitted]             
2.bundle.js   552099       2  [emitted]             
3.bundle.js  1848882       3  [emitted]             
4.bundle.js  2413145       4  [emitted] 

That's 5.9MB in total at the moment, even though the unuglified r.js build is 3.8MB. That's because there's a lot of code duplication in chunks 1, 2, 3, 4. I'd like to build all modules that are shared at least once into the parent bundle, but haven't figured how to do that yet.

@sokra

This comment has been minimized.

Member

sokra commented Feb 12, 2014

There was a typo in the documentation. It's output.pathinfo with a lowercase i.

SourceMaps are a bit slow, because every line in source need to tracked and the resulting SourceMaps are pretty big. They are only updated when a chunk has changed, so smaller chunk may result in fast rebuilds.

The eval devtool is much faster, because it only need to stringify the source and nothing need to be tracked.

I use the eval devtool for development and SourceMaps for production. With pathinfo = true the eval tool is enough for me (if you develop in a non-js language, i.e. coffeescript, you may need SourceMaps).

Maybe #169 can provide better performance for SourceMaps...


Generally it's better to have a smaller initial chunk (and not move modules from on-demand chunks into the initial chunk), because this chunk affects your initial loading time.

There is an function to include a module into a bundle but don't execute it: require.include.

A leftover chunk (#70) may be a better option to solve this duplication, but this isn't supported by webpack. You can do it manually, by wrapping every require.ensure/AMD require in an additional require.ensure which contains all common modules.

@jhnns

This comment has been minimized.

Member

jhnns commented Feb 12, 2014

devtool: "eval" for development is ok. I don't notice the numbers instead of strings anymore (that's usually not the code you're trying to debug anyway).

@sokra

This comment has been minimized.

Member

sokra commented Feb 13, 2014

@KidkArolis Could you test the performance of devtool: "eval-source-map" for your project? (webpack@1.0.0-rc8)

@sokra sokra added this to the 1.0 milestone Feb 13, 2014

@KidkArolis

This comment has been minimized.

KidkArolis commented Feb 13, 2014

Wow, that was quick! Will do once I get home from London JS Conf.
On Feb 13, 2014 10:47 AM, "Tobias Koppers" notifications@github.com wrote:

@KidkArolis https://github.com/KidkArolis Could you test the
performance of devtool: "eval-source-map" for your project?

Reply to this email directly or view it on GitHubhttps://github.com//issues/168#issuecomment-34967181
.

@KidkArolis

This comment has been minimized.

KidkArolis commented Feb 14, 2014

Works very well - the rebuilds with eval-source-map seem to be as fast as with eval.

Chrome (beta) logs data:url requests in the network panel and because each module now has it's own sourcemap data:url, 450 of those in my case needlessly slow down the dev tools. It's possible to filter the data:urls out which helps a bit. Not sure if that just slows down the network panel or perhaps in general it's slightly slower to load each sourcemap from the data:urls individually. But anyways, that's up to Chrome to improve, the eval-source-map feature works well - neat idea!

@jhnns

This comment has been minimized.

Member

jhnns commented Feb 14, 2014

Nice, @sokra 👍

@sokra sokra closed this Feb 19, 2014

@nicolashery

This comment has been minimized.

nicolashery commented Aug 15, 2014

This is really useful, thanks @sokra!

I couldn't find it in the documentation though. Did I miss it?

@sokra

This comment has been minimized.

@sebastienbarre

This comment has been minimized.

Contributor

sebastienbarre commented Mar 17, 2015

Switching from source-map to eval-source-map it looks like the CSS source maps are not being bundled by the ExtractTextPlugin. Is it expected? Thanks

@sokra

This comment has been minimized.

Member

sokra commented Apr 8, 2015

Switching from source-map to eval-source-map it looks like the CSS source maps are not being bundled by the ExtractTextPlugin. Is it expected? Thanks

yes.

webpack 1.8 now offers an additional SourceMap option which is pretty fast but generates SourceMaps: devtool: "cheap-source-map" and devtool: "cheap-module-source-map" (plus eval- variants).

Here is a overview: http://webpack.github.io/docs/configuration.html#devtool

@sebastienbarre

This comment has been minimized.

Contributor

sebastienbarre commented Apr 8, 2015

Thanks. I'm confused though. How do you use both the devtool option and the SourceMapDevToolPlugin? I have to use the plugin because of the exclude parameter I contributed in PR #897.

  1. If I do this alone:
config.devtool = 'source-map';

My sourceMaps are generated fine. There is a big 2.2 MB vendors.js.map generated for my vendors chunk that I don't need, and a 177KB app.js.map I want to keep around.

  1. Now that webpack 1.8.0 has #897 I can exclude the vendors sourceMap:
config.plugins.push(new webpack.SourceMapDevToolPlugin({
   filename: '[file].map',
   exclude: ['vendors.js']
}));

Great! I end up with only my app sourceMap.

  1. Now if I use this below to get these fast sourceMaps you mentioned:
config.devtool = 'cheap-source-map';

I end up with a 113KB apps.js.map (nice) but still a big vendors.js.map.

How do I combine both 2) and 3)? How can I exclude my vendors chunk while using cheap sourceMap for my app chunk?

If I use both a the same time:

config.devtool = 'cheap-source-map';
config.plugins.push(new webpack.SourceMapDevToolPlugin({
   filename: '[file].map',
   exclude: ['vendors.js']
}));

I end up with nothing usable, my vendors sourceMap is still there, and my app sourceMap appears corrupted (only 3KB). It is not clear to me what the relationship is between devtool and the plugin itself. How do you specify in the plugin options which type of source map you want (cheap, eval, etc)? Or is there a plugin for each type of devtool value? And if that's the case, do I need to add this exclude flag for all the plugins?

Thanks

@sokra

This comment has been minimized.

Member

sokra commented Apr 8, 2015

The devtool option maps to the SourceMapDevToolPlugin (see here). If you want to apply additional option for the plugin, you need to use the plugin version and omit the devtool options. Using both doesn't make sense.

The cheap SourceMaps map to columns: false:

new webpack.SourceMapDevToolPlugin({
   filename: '[file].map',
   exclude: ['vendors.js'],
   columns: false, // no columns in SourceMaps
   module: false / true // use SourceMaps from loaders 
})

I'll add documentation for the SourceMapDevToolPlugin to the list of plugins.

@sebastienbarre

This comment has been minimized.

Contributor

sebastienbarre commented Apr 17, 2015

Thanks @sokra. cheap-source-map is indeed as fast as eval for me. I posted some numbers below. Unfortunately, I must be doing something wrong because I can't see any useful difference between eval and source-map at this point. None of these options ever points back at my original ES6 code. And most of the source-map variants actually reports the correct line:column location in the my ES6 code, but bring the transformed-to-ES5 code in the Chrome debug window, where that location does not match. Puzzling.

The numbers below are for a small test app, where webpack seems to be going through 470+ modules during build. My code uses ES6 and React extensively (and some ES7), therefore babel comes into play to transform everything back to ES5. I generate a small app.js bundle for my own code, and a much larger vendors.js for everything else found under node_modules.

In the scenarios below I introduced an error that shows up at runtime. I then click on the error message in Chrome's console, and check where that leads me (i.e. is the line:column right on the error, and with respect to which file). I also report how long it took to run webpack (averaged over 3 tries).

eval - Each module is executed with eval and //@ sourceURL.

  • time: 8.4s
  • vendors.js: 1.61 MB
  • kinda OK. Points at the correct line:column but inside what looks like the transformed-to-ES5 file (after babel's ES6-to-ES5 transformation, that is), not my original ES6 file.

source-map - A SourceMap is emitted.

  • time: 10s
  • vendors.js: 1.51 MB
  • vendors.map: 1.72 MB
  • NOT OK. Reports the correct line:column w/ respect to the original ES6 source file but shows the transformed-to-ES5 file instead (mozilla/source-map issue?).

hidden-source-map - Same as source-map, but doesn’t add a reference comment to the bundle.

  • time: 9.8s
  • vendors.js: 1.51 MB
  • vendors.map: 1.72 MB
  • kinda OK. Behaves like eval (to me), points at the correct line:column but inside the resulting huge transformed-to-ES5 bundle/module file (vendor.js), not the original, specific ES6 file.

inline-source-map - A SourceMap is added as DataUrl to the JavaScript file.

  • time: 9.9s
  • vendors.js: 3.81 MB
  • NOT OK. Behaves like source-map (but inlined).

eval-source-map - Each module is executed with eval and a SourceMap is added as DataUrl to the eval.

  • time: 9.8s
  • vendors.js: 3.88 MB
  • NOT OK. Behaves like source-map (but inline/eval'ed).

cheap-source-map - (new in 1.8) A SourceMap is emitted. Faster version.

  • time: 8.4s
  • vendors.js: 1.50 MB
  • vendors.map: 1.71 MB
  • kinda OK. Behaves like eval (i.e. points at a transformed-to-ES5 file, not the original ES6 file).
@sokra

This comment has been minimized.

Member

sokra commented Apr 20, 2015

And most of the source-map variants actually reports the correct line:column location in the my ES6 code, but bring the ES5-transformed code in the Chrome debug window, where that location does not match. Puzzling.

You need to enabled SourceMaps in the Chrome DevTool settings.

@sebastienbarre

This comment has been minimized.

Contributor

sebastienbarre commented Apr 21, 2015

It is enabled. I'm wondering if they are read properly though. Maybe something to do with publicPath?

On Apr 21, 2015, at 04:45, Tobias Koppers notifications@github.com wrote:

And most of the source-map variants actually reports the correct line:column location in the my ES6 code, but bring the ES5-transformed code in the Chrome debug window, where that location does not match. Puzzling.

You need to enabled SourceMaps in the Chrome DevTool settings.


Reply to this email directly or view it on GitHub.

@sokra

This comment has been minimized.

Member

sokra commented Apr 21, 2015

I don't know. SourceMaps work fine for me. You should be able to see the SourceMap requests in the Network tab.

@sebastienbarre

This comment has been minimized.

Contributor

sebastienbarre commented May 6, 2015

@sokra could you elaborate on "work fine for me"? I can see the SourceMap requests in the Network tab. Actually if I turn SourceMaps off in DevTool settings, and I click on the error in the console, it brings me to the bundle file (app.js). If I turn it on, it leads me to "something" that has the same name as the original source file the error is in (aka HomePage.react.js, not the bundle app.js file). So it seems to be doing something. BUT, this is not my real original HomePage.react.js, it's a version of it after the babel/JSX transformation, i.e. instead of seeing, for example:

class HomePage extends PureRender {
  render() {
    foo = foo / 0; // THE ERROR
    return (
      <div className="home">
[...]

I see:

var HomePage = (function (_PureRender) {
  function HomePage() {
    _classCallCheck(this, HomePage);
    if (_PureRender != null) {
      _PureRender.apply(this, arguments);
    }
  }
  _inherits(HomePage, _PureRender);
  _createClass(HomePage, [{
    key: 'render',
    value: function render() {
      foo = foo / 0;
      return React.createElement('div', { className: 'home' }, [...]

Is that the behavior you are seeing, and is it expected? I was expecting SourceMaps to point at the original file, the real one, in ES6, before any transformation (i.e. the one I see in my code editor).

@sokra

This comment has been minimized.

Member

sokra commented May 6, 2015

Which devtool? (only cheap-module-source-map and source-map, optionally plus eval- support module mappings)
Which loaders are applied for the file? (all loaders applied must support SourceMaps)

@sebastienbarre

This comment has been minimized.

Contributor

sebastienbarre commented May 6, 2015

Thanks @sokra. I've to admit, I do not know what module mappings are, can't find documentation.

When I use devtool = 'source-map' the correct line number is reported from the original ES6 file, BUT the incorrect transformed-to-ES5 file is loaded by Chrome. Which is unusable for debugging, since they don't match.

When I look at the SourceMap file, I can see references like "webpack:///./src/js/pages/HomePage.react.js". Inside the devTool panel in Chrome, in the Sources tab, I can see a webpack:// entry in the Sources tree; if I navigate that tree, I can find the src/js/pages/HomePage.react.js entry, but it is the transformed file, not the original file.

The loader being used for .react.js files is babel-loader?stage=0, i.e. Babel Loader.

@lorenzomigliorero

This comment has been minimized.

lorenzomigliorero commented Sep 3, 2016

stats: 'errors-only' don't work for me...
I prefer a simple console.warn = function() {} to prevent console message on client...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment