New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webpack incremental builds are slow #616

Open
quicksnap opened this Issue Nov 25, 2015 · 124 comments

Comments

Projects
None yet
@quicksnap
Collaborator

quicksnap commented Nov 25, 2015

On my machine, incremental builds in dev mode take anywhere from 1.5s - 5s.

I'm going to spend some time toying with webpack to see what's up, but I'd appreciate anyone else's ideas on what's going wrong.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Nov 25, 2015

Collaborator

I'm able to drop the rebuild time significantly without bootstrap/font-awesome loaders in the main entry. Going to look into splitting it into a separate chunk.

Collaborator

quicksnap commented Nov 25, 2015

I'm able to drop the rebuild time significantly without bootstrap/font-awesome loaders in the main entry. Going to look into splitting it into a separate chunk.

@trueter

This comment has been minimized.

Show comment
Hide comment
@trueter

trueter Nov 26, 2015

Contributor

Sounds great!

Contributor

trueter commented Nov 26, 2015

Sounds great!

@wwwfreedom

This comment has been minimized.

Show comment
Hide comment
@wwwfreedom

wwwfreedom Nov 26, 2015

I'm getting slow build time as well. Looking forward to your split set up.

wwwfreedom commented Nov 26, 2015

I'm getting slow build time as well. Looking forward to your split set up.

@luankefei

This comment has been minimized.

Show comment
Hide comment
@luankefei

luankefei Nov 26, 2015

I am the same. it happened looks like webpack-hot-middleware didn't work.

luankefei commented Nov 26, 2015

I am the same. it happened looks like webpack-hot-middleware didn't work.

@bdefore

This comment has been minimized.

Show comment
Hide comment
@bdefore

bdefore Nov 26, 2015

Collaborator

@quicksnap if you set your devtool to inline-eval-cheap-source-map in dev.config.js i've noticed around 75% speed improvement on rebuilds, 10% improvement on initial builds. there's some discussion around why source mapping is expensive in recent versions, and it looks like it might be due to css-loader's implementation of css modules. you may also want to upgrade node-sass to 3.4.x. looks like the example project still has 3.3.3 which @Phoenixmatrix identified as having poor performance: webpack-contrib/sass-loader#176

Collaborator

bdefore commented Nov 26, 2015

@quicksnap if you set your devtool to inline-eval-cheap-source-map in dev.config.js i've noticed around 75% speed improvement on rebuilds, 10% improvement on initial builds. there's some discussion around why source mapping is expensive in recent versions, and it looks like it might be due to css-loader's implementation of css modules. you may also want to upgrade node-sass to 3.4.x. looks like the example project still has 3.3.3 which @Phoenixmatrix identified as having poor performance: webpack-contrib/sass-loader#176

@cjhveal

This comment has been minimized.

Show comment
Hide comment
@cjhveal

cjhveal Nov 26, 2015

If I'm not mistaken, PR #614 upgraded node-sass to ^3.4.2.

cjhveal commented Nov 26, 2015

If I'm not mistaken, PR #614 upgraded node-sass to ^3.4.2.

@erikras

This comment has been minimized.

Show comment
Hide comment
@erikras

erikras Nov 27, 2015

Owner

Dupes #594

Owner

erikras commented Nov 27, 2015

Dupes #594

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

Weird.. I thought I commented a reply to @bdefore:

Modifying or even removing the devtool doesn't do much for me. In my case, it's nearly all time due to the bootstrap-loader plugin. When I remove that, my rebuild time drops from ~4000ms to <500ms.

It would be nice to understand exactly what is going on. I tried digging into the plugin itself, but any modifications I made would not yield positive results, other than causing it not to generate the SASS file which @imports all the bootstrap stuff.

I have a feeling it's not properly caching due to its usage of a pitch loader. I'm not sure though, and it's tough to get help or even form an educated question.

Personally, I may just remove bootstrap loader and use a static build of bootstrap. I'm not sure the best route for this project.

Would love it if someone could see why bootstrap-loader isn't caching on rebuilds. Or anything to drop rebuild times <1000ms

Collaborator

quicksnap commented Dec 1, 2015

Weird.. I thought I commented a reply to @bdefore:

Modifying or even removing the devtool doesn't do much for me. In my case, it's nearly all time due to the bootstrap-loader plugin. When I remove that, my rebuild time drops from ~4000ms to <500ms.

It would be nice to understand exactly what is going on. I tried digging into the plugin itself, but any modifications I made would not yield positive results, other than causing it not to generate the SASS file which @imports all the bootstrap stuff.

I have a feeling it's not properly caching due to its usage of a pitch loader. I'm not sure though, and it's tough to get help or even form an educated question.

Personally, I may just remove bootstrap loader and use a static build of bootstrap. I'm not sure the best route for this project.

Would love it if someone could see why bootstrap-loader isn't caching on rebuilds. Or anything to drop rebuild times <1000ms

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

Perhaps it's simply that bootstrap generates a ton of CSS, and css-loader chokes on that.

I will try bypassing css-loader for bootstrap and see what happens.

Collaborator

quicksnap commented Dec 1, 2015

Perhaps it's simply that bootstrap generates a ton of CSS, and css-loader chokes on that.

I will try bypassing css-loader for bootstrap and see what happens.

@Phoenixmatrix

This comment has been minimized.

Show comment
Hide comment
@Phoenixmatrix

Phoenixmatrix Dec 1, 2015

I don't know much on bootstrap-loader, but saw this since i got tagged on it. Its possible what's below is irrelevant, but I'll pitch it in anyway. Though it sounds like its just the loader not caching, as as mentioned.

I dont know what boostrap loader does, but do keep in mind that SASS doesn't get incrementally compiled the same way JS does. If you require(...) SASS, it creates a css "entry point" so to speak, but if inside that SASS, you @import other stuff, that's all part of the same "sass build". Sass builds are not incremental, and all SASS within a single "entry point" will get recompiled every time.

node-sass <3.4 is slow, and css-loader after 0.14.5 is BRUTALLY slow (to the point of being unusable). Since all of your CSS may get recompiled if its all in the same CSS entry point, you can quickly end up in CSS builds that take longer than rebuilding your entire JS from scratch, even though you only added 1 character to 1 CSS file.

Note that splitting your CSS by requiring it from JS isn't always a good answer in SASS because of the lack of @import (reference) like LESS does...you may end up duplicating a lot of stuff if you're not careful.

tl;dr: CSS builds using the @import mechanic are not incremental and cause a full rebuild of that part. You probably want a static bootstrap build, and @import only variables and mixins as need be (but nothing else!)

Phoenixmatrix commented Dec 1, 2015

I don't know much on bootstrap-loader, but saw this since i got tagged on it. Its possible what's below is irrelevant, but I'll pitch it in anyway. Though it sounds like its just the loader not caching, as as mentioned.

I dont know what boostrap loader does, but do keep in mind that SASS doesn't get incrementally compiled the same way JS does. If you require(...) SASS, it creates a css "entry point" so to speak, but if inside that SASS, you @import other stuff, that's all part of the same "sass build". Sass builds are not incremental, and all SASS within a single "entry point" will get recompiled every time.

node-sass <3.4 is slow, and css-loader after 0.14.5 is BRUTALLY slow (to the point of being unusable). Since all of your CSS may get recompiled if its all in the same CSS entry point, you can quickly end up in CSS builds that take longer than rebuilding your entire JS from scratch, even though you only added 1 character to 1 CSS file.

Note that splitting your CSS by requiring it from JS isn't always a good answer in SASS because of the lack of @import (reference) like LESS does...you may end up duplicating a lot of stuff if you're not careful.

tl;dr: CSS builds using the @import mechanic are not incremental and cause a full rebuild of that part. You probably want a static bootstrap build, and @import only variables and mixins as need be (but nothing else!)

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

@Phoenixmatrix thanks. The problem with this setup is that whenever we change any of our JS files, it triggers an entire rebuild, including the CSS. I could not figure a way to tell webpack not to rebuild bootstrap-sass every time. It is doing so here: https://github.com/erikras/react-redux-universal-hot-example/blob/master/webpack/dev.config.js#L54

I tried splitting out into a different chunk, but no luck. My webpack chops are mediocre.

I'm tooling around with bootstrap-sass-loader code now, but I still am not entirely sure where the slowness is being generated.

Collaborator

quicksnap commented Dec 1, 2015

@Phoenixmatrix thanks. The problem with this setup is that whenever we change any of our JS files, it triggers an entire rebuild, including the CSS. I could not figure a way to tell webpack not to rebuild bootstrap-sass every time. It is doing so here: https://github.com/erikras/react-redux-universal-hot-example/blob/master/webpack/dev.config.js#L54

I tried splitting out into a different chunk, but no luck. My webpack chops are mediocre.

I'm tooling around with bootstrap-sass-loader code now, but I still am not entirely sure where the slowness is being generated.

@Phoenixmatrix

This comment has been minimized.

Show comment
Hide comment
@Phoenixmatrix

Phoenixmatrix Dec 1, 2015

Yeah, i realized that afterward rereading your comment. Whoops! For sure upgrading to node 3.4 should speed that up since compiling bootstrap shouldn't take very long at all, but obviously that only fixes symptoms (did you know node-sass is like 2-3 times faster on Linux than MacOSX? fun stuff).

Will take a look at the bootstrap sass loader when I get the chance.

Phoenixmatrix commented Dec 1, 2015

Yeah, i realized that afterward rereading your comment. Whoops! For sure upgrading to node 3.4 should speed that up since compiling bootstrap shouldn't take very long at all, but obviously that only fixes symptoms (did you know node-sass is like 2-3 times faster on Linux than MacOSX? fun stuff).

Will take a look at the bootstrap sass loader when I get the chance.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

Thanks for helping out! Appreciate it.

Be sure to change the devtool to something fast like inline-eval-cheap-source-map. I forgot to do that and it was screwing up my benchmarks.

Collaborator

quicksnap commented Dec 1, 2015

Thanks for helping out! Appreciate it.

Be sure to change the devtool to something fast like inline-eval-cheap-source-map. I forgot to do that and it was screwing up my benchmarks.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

Ok! I just changed devtool to inline-eval-cheap-source-map, and then edited bootstrap-sass-loader to this: quicksnap/bootstrap-sass-loader@6e60e0f (switch out css-loader for raw-loader)

This significantly speeds things up for me.

Collaborator

quicksnap commented Dec 1, 2015

Ok! I just changed devtool to inline-eval-cheap-source-map, and then edited bootstrap-sass-loader to this: quicksnap/bootstrap-sass-loader@6e60e0f (switch out css-loader for raw-loader)

This significantly speeds things up for me.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

I think it still would be an even better improvement if we could get webpack to avoid rebuilding bootstrap unless the bootstrap config file changes.

Collaborator

quicksnap commented Dec 1, 2015

I think it still would be an even better improvement if we could get webpack to avoid rebuilding bootstrap unless the bootstrap config file changes.

@Phoenixmatrix

This comment has been minimized.

Show comment
Hide comment
@Phoenixmatrix

Phoenixmatrix Dec 1, 2015

Yeah, so obviously that part is still broken. But also, yet another reason
css-loader needs to be fixed. Its so completely broken performance wise
right now, as per webpack-contrib/css-loader#124

In our codebase its unusable (css build takes minutes, so we have to use
raw-loader)

Phoenixmatrix commented Dec 1, 2015

Yeah, so obviously that part is still broken. But also, yet another reason
css-loader needs to be fixed. Its so completely broken performance wise
right now, as per webpack-contrib/css-loader#124

In our codebase its unusable (css build takes minutes, so we have to use
raw-loader)

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

I hope it is improved by the time our css grows. I really like CSS Modules...

Collaborator

quicksnap commented Dec 1, 2015

I hope it is improved by the time our css grows. I really like CSS Modules...

@mmahalwy

This comment has been minimized.

Show comment
Hide comment
@mmahalwy

mmahalwy Dec 1, 2015

@quicksnap seems like you're suggesting to avoid bootstrap-sass-loader? That seems to be the bottleneck?

mmahalwy commented Dec 1, 2015

@quicksnap seems like you're suggesting to avoid bootstrap-sass-loader? That seems to be the bottleneck?

@bdefore

This comment has been minimized.

Show comment
Hide comment
@bdefore

bdefore Dec 1, 2015

Collaborator

i had a chat with the shakacode folks this morning and discovered that bootstrap-sass-loader is being rewritten and will soon be deprecated. the rewritten version is here, though it hasn't been touched in a couple weeks: https://github.com/shakacode/bootstrap-loader/tree/alex/bootstrap-4

Collaborator

bdefore commented Dec 1, 2015

i had a chat with the shakacode folks this morning and discovered that bootstrap-sass-loader is being rewritten and will soon be deprecated. the rewritten version is here, though it hasn't been touched in a couple weeks: https://github.com/shakacode/bootstrap-loader/tree/alex/bootstrap-4

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

@mmahalwy I'm not recommending avoiding it--I'll try to issue a PR to bootstrap-sass-loader soon. The change is minor. I don't see any problems using raw-loader over css-loader in bootstrap-sass-loader.

Collaborator

quicksnap commented Dec 1, 2015

@mmahalwy I'm not recommending avoiding it--I'll try to issue a PR to bootstrap-sass-loader soon. The change is minor. I don't see any problems using raw-loader over css-loader in bootstrap-sass-loader.

@Phoenixmatrix

This comment has been minimized.

Show comment
Hide comment
@Phoenixmatrix

Phoenixmatrix Dec 1, 2015

Raw loader will not rewrite image and font URLs if you hash them. I don't
know if it affects bootstrap in any way.

On Tue, Dec 1, 2015, 1:24 PM Dan Schuman notifications@github.com wrote:

@mmahalwy https://github.com/mmahalwy I'm not recommending avoiding
it--I'll try to issue a PR to bootstrap-sass-loader soon. The change is
minor. I don't see any problems using raw-loader over css-loader in
bootstrap-sass-loader.


Reply to this email directly or view it on GitHub
#616 (comment)
.

-Francois Ward

Phoenixmatrix commented Dec 1, 2015

Raw loader will not rewrite image and font URLs if you hash them. I don't
know if it affects bootstrap in any way.

On Tue, Dec 1, 2015, 1:24 PM Dan Schuman notifications@github.com wrote:

@mmahalwy https://github.com/mmahalwy I'm not recommending avoiding
it--I'll try to issue a PR to bootstrap-sass-loader soon. The change is
minor. I don't see any problems using raw-loader over css-loader in
bootstrap-sass-loader.


Reply to this email directly or view it on GitHub
#616 (comment)
.

-Francois Ward

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

@Phoenixmatrix good point. I'll check on that, but I have a feeling it could cause issues now.

Collaborator

quicksnap commented Dec 1, 2015

@Phoenixmatrix good point. I'll check on that, but I have a feeling it could cause issues now.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

I'll also look into passing params to css-loader to disable some things, or to downgrade css-loader in that repository.

Collaborator

quicksnap commented Dec 1, 2015

I'll also look into passing params to css-loader to disable some things, or to downgrade css-loader in that repository.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 1, 2015

Collaborator

Or, having css-loader be a peer dependency.

Collaborator

quicksnap commented Dec 1, 2015

Or, having css-loader be a peer dependency.

@trueter

This comment has been minimized.

Show comment
Hide comment
@trueter

trueter Dec 2, 2015

Contributor

@quicksnap: I tried splitting out into a different chunk, but no luck. My webpack chops are mediocre.

What was the reason you gave up on this route? Having bootstrap outside the main chunk might circumvent the recompile issue.

Contributor

trueter commented Dec 2, 2015

@quicksnap: I tried splitting out into a different chunk, but no luck. My webpack chops are mediocre.

What was the reason you gave up on this route? Having bootstrap outside the main chunk might circumvent the recompile issue.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 2, 2015

Collaborator

@trueter yes--I couldn't figure how to prevent bootstra-sass-loader from being triggered every rebuild. I did get it into a separate chunk successfully, but it would always rebuild all chunks AFAIK (it would say 'emitted' next to the vendor chunk I created).

Collaborator

quicksnap commented Dec 2, 2015

@trueter yes--I couldn't figure how to prevent bootstra-sass-loader from being triggered every rebuild. I did get it into a separate chunk successfully, but it would always rebuild all chunks AFAIK (it would say 'emitted' next to the vendor chunk I created).

@mmahalwy

This comment has been minimized.

Show comment
Hide comment
@mmahalwy

mmahalwy Dec 4, 2015

@quicksnap any luck on this so far?

mmahalwy commented Dec 4, 2015

@quicksnap any luck on this so far?

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 4, 2015

Collaborator

@mmahalwy I haven't messed with adjusting bootstrap-sass-loader's properties yet. I'm also not sure if bootstrap-sass has any image URLs in it, so it may actually be good with raw-loader, which would be hella faster.

I'm working on stuff right now that doesn't involve dev mode, but next time I get hit with a 4sec rebuild I'll probably dig in.

Collaborator

quicksnap commented Dec 4, 2015

@mmahalwy I haven't messed with adjusting bootstrap-sass-loader's properties yet. I'm also not sure if bootstrap-sass has any image URLs in it, so it may actually be good with raw-loader, which would be hella faster.

I'm working on stuff right now that doesn't involve dev mode, but next time I get hit with a 4sec rebuild I'll probably dig in.

@sars

This comment has been minimized.

Show comment
Hide comment
@sars

sars Dec 12, 2015

Contributor

@quicksnap what's your build time now?
I've just launched this example and it takes 3.5-5 sec for me for each build...
it really slows down development..

Did you figure out how to speed it up?

Contributor

sars commented Dec 12, 2015

@quicksnap what's your build time now?
I've just launched this example and it takes 3.5-5 sec for me for each build...
it really slows down development..

Did you figure out how to speed it up?

@trueter

This comment has been minimized.

Show comment
Hide comment
@trueter

trueter Dec 15, 2015

Contributor

@quicksnap could you create a pr with your code split work in progress? stumbled across this: http://tech.trivago.com/2015/12/15/parallel-webpack/

Contributor

trueter commented Dec 15, 2015

@quicksnap could you create a pr with your code split work in progress? stumbled across this: http://tech.trivago.com/2015/12/15/parallel-webpack/

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 15, 2015

Collaborator

@sars @trueter Here's what will speed things up for me: https://github.com/erikras/react-redux-universal-hot-example/compare/master...quicksnap:foobar?expand=1

However, this could break things with bootstrap: raw-loader will not extract url() paths, and I'm unsure if bootstrap utilizes this. I do know they have some gylphicon stuff in there, so, it very well may cause issues.

Collaborator

quicksnap commented Dec 15, 2015

@sars @trueter Here's what will speed things up for me: https://github.com/erikras/react-redux-universal-hot-example/compare/master...quicksnap:foobar?expand=1

However, this could break things with bootstrap: raw-loader will not extract url() paths, and I'm unsure if bootstrap utilizes this. I do know they have some gylphicon stuff in there, so, it very well may cause issues.

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Dec 15, 2015

Collaborator

Whoops--you'll also need to npm i --save raw-loader in order for that to work.

Collaborator

quicksnap commented Dec 15, 2015

Whoops--you'll also need to npm i --save raw-loader in order for that to work.

@stevoland

This comment has been minimized.

Show comment
Hide comment
@stevoland

stevoland Jan 20, 2016

Collaborator

I've just updated a project with a similar Webpack config (CSS Modules, node-sass etc) from Babel 5 to 6 (with es2015-loose). Incremental builds went from a horrible 4-6sec to a snappy 1-2sec. Might be worth looking into.

Collaborator

stevoland commented Jan 20, 2016

I've just updated a project with a similar Webpack config (CSS Modules, node-sass etc) from Babel 5 to 6 (with es2015-loose). Incremental builds went from a horrible 4-6sec to a snappy 1-2sec. Might be worth looking into.

@trueter

This comment has been minimized.

Show comment
Hide comment
@trueter

trueter Jan 20, 2016

Contributor

Does it include bootstrap? Is it a public repo you can share?

Contributor

trueter commented Jan 20, 2016

Does it include bootstrap? Is it a public repo you can share?

@ozooner

This comment has been minimized.

Show comment
Hide comment
@ozooner

ozooner Jan 20, 2016

He said it's a project with similar configuration...
There's an ongoing babel 6 upgrade issue #488

ozooner commented Jan 20, 2016

He said it's a project with similar configuration...
There's an ongoing babel 6 upgrade issue #488

@isaacl

This comment has been minimized.

Show comment
Hide comment
@isaacl

isaacl Feb 20, 2016

FWIW, disabling eslint-loader cut our build time by about 2x. eslint is just really slow, and the semantics of the loader api make it difficult to run eslint efficiently. I wrote a script (py | bash)** as an alternative but for an example app it should be packaged, etc. Some discussion here: webpack-contrib/eslint-loader#82.

** not thoroughly tested!

isaacl commented Feb 20, 2016

FWIW, disabling eslint-loader cut our build time by about 2x. eslint is just really slow, and the semantics of the loader api make it difficult to run eslint efficiently. I wrote a script (py | bash)** as an alternative but for an example app it should be packaged, etc. Some discussion here: webpack-contrib/eslint-loader#82.

** not thoroughly tested!

@quicksnap

This comment has been minimized.

Show comment
Hide comment
@quicksnap

quicksnap Feb 20, 2016

Collaborator

That's good to know!

Collaborator

quicksnap commented Feb 20, 2016

That's good to know!

@vjpr

This comment has been minimized.

Show comment
Hide comment
@vjpr

vjpr Feb 25, 2016

If you have require('colors') anywhere in your codebase or any of your dependencies, and also use bootstrap-loader, check out the crazy bug I found here: reworkcss/css#88

My incremental build went from 14s to 7s.

For now just modify node_modules/resolve-url-loaders/node_modules/rework/node_modules/css/lib/parse/index.js directly to see if if helps.

vjpr commented Feb 25, 2016

If you have require('colors') anywhere in your codebase or any of your dependencies, and also use bootstrap-loader, check out the crazy bug I found here: reworkcss/css#88

My incremental build went from 14s to 7s.

For now just modify node_modules/resolve-url-loaders/node_modules/rework/node_modules/css/lib/parse/index.js directly to see if if helps.

@vjpr

This comment has been minimized.

Show comment
Hide comment
@vjpr

vjpr Feb 25, 2016

I have been trying to work this out for my codebase (40s build / 10s incremental) and the issue it seems is a lack of easily accessible logging.

In Compilation.js, Compile#addModuleDependencies is called recursively and takes up the majority of the incremental build, while the build-module event is not firing. So my issue seems to be that it tries to build the dependency tree everytime without caching it, which is slow. I have asked a question here: webpack/webpack#2102 to confirm this.

Some ideas if dependency resolution trees are not cacheable:

  • Don't make webpack look inside every file in node_modules for dependencies. This can be done using the dist builds which most modules have. This makes debugging more difficult though :(. This can be done using noParse.
  • Identify which modules are expensive to resolve dependencies, and don't parse them. It appears that dependency resolution is asynchronous/parallel making it hard to isolate a single module resolution time (or is async used to do it in some order). If we could run this synchronously, we could easily see which modules take a long time to parse.
  • DLLCachePlugin - I'm very hopeful about this...I think it could solve my problem completely.

vjpr commented Feb 25, 2016

I have been trying to work this out for my codebase (40s build / 10s incremental) and the issue it seems is a lack of easily accessible logging.

In Compilation.js, Compile#addModuleDependencies is called recursively and takes up the majority of the incremental build, while the build-module event is not firing. So my issue seems to be that it tries to build the dependency tree everytime without caching it, which is slow. I have asked a question here: webpack/webpack#2102 to confirm this.

Some ideas if dependency resolution trees are not cacheable:

  • Don't make webpack look inside every file in node_modules for dependencies. This can be done using the dist builds which most modules have. This makes debugging more difficult though :(. This can be done using noParse.
  • Identify which modules are expensive to resolve dependencies, and don't parse them. It appears that dependency resolution is asynchronous/parallel making it hard to isolate a single module resolution time (or is async used to do it in some order). If we could run this synchronously, we could easily see which modules take a long time to parse.
  • DLLCachePlugin - I'm very hopeful about this...I think it could solve my problem completely.
@alecperkey

This comment has been minimized.

Show comment
Hide comment
@alecperkey

alecperkey May 25, 2016

This commit I've removed some devDependencies which probably dont need to be in the vendorArray.. but I'm unsure which other ones are unnecessary.

And the npm script 'client-prepare' I have no idea what 'invalid data' error is. If i run the webpack command directly, it works, but I need the better npm run for the MAKE_DLL=1 variable on windows.

alecperkey commented May 25, 2016

This commit I've removed some devDependencies which probably dont need to be in the vendorArray.. but I'm unsure which other ones are unnecessary.

And the npm script 'client-prepare' I have no idea what 'invalid data' error is. If i run the webpack command directly, it works, but I need the better npm run for the MAKE_DLL=1 variable on windows.

@slavab89

This comment has been minimized.

Show comment
Hide comment
@slavab89

slavab89 May 26, 2016

@alecperkey Might be some windows and better npm thing that does not work together? Maybe that's not how you define a variable for the script to use?

Regarding the vendor array. Basically anything that you use on the client side can go there. For example stuff like express should not go there because that's sever side (i think it will also not let you and wont work) but anything client related can go there. I've just gone over all the import statements of the client files and took the packages

slavab89 commented May 26, 2016

@alecperkey Might be some windows and better npm thing that does not work together? Maybe that's not how you define a variable for the script to use?

Regarding the vendor array. Basically anything that you use on the client side can go there. For example stuff like express should not go there because that's sever side (i think it will also not let you and wont work) but anything client related can go there. I've just gone over all the import statements of the client files and took the packages

@alecperkey

This comment has been minimized.

Show comment
Hide comment
@alecperkey

alecperkey May 31, 2016

@slavab89, thanks for the suggestion. It feels so close!

Not sure what the better npm run problem is... cause it works fine on a different repo for setting env variables this way. However, I've just made a 'quick hack' workaround for now.

I updated the vendor array with your method suggested above.
There is just one error remaining -- when generating the app DLL bundle. It is in the ERROR LOG.txt

[TypeError: The plugin ["react-transform",{"transforms":[{"transform":"react-transform-hmr","imports":["react"],"locals":["module"]}]}] didn't export a Plugin instance]

If anyone can help, see commit: alecperkey@2ecde40

alecperkey commented May 31, 2016

@slavab89, thanks for the suggestion. It feels so close!

Not sure what the better npm run problem is... cause it works fine on a different repo for setting env variables this way. However, I've just made a 'quick hack' workaround for now.

I updated the vendor array with your method suggested above.
There is just one error remaining -- when generating the app DLL bundle. It is in the ERROR LOG.txt

[TypeError: The plugin ["react-transform",{"transforms":[{"transform":"react-transform-hmr","imports":["react"],"locals":["module"]}]}] didn't export a Plugin instance]

If anyone can help, see commit: alecperkey@2ecde40

@jaraquistain

This comment has been minimized.

Show comment
Hide comment
@jaraquistain

jaraquistain Jun 2, 2016

I wanted to use this thread as a starting point for improving my webpack config and it occurred to me that it could be useful if people who have spent a significant time on this subject maybe posted a gist of their webpack config for reference. Anyone willing to throw one up?

jaraquistain commented Jun 2, 2016

I wanted to use this thread as a starting point for improving my webpack config and it occurred to me that it could be useful if people who have spent a significant time on this subject maybe posted a gist of their webpack config for reference. Anyone willing to throw one up?

@ianks

This comment has been minimized.

Show comment
Hide comment
@ianks

ianks Jun 2, 2016

Here is my dll.config.js file:

'use strict';

const _ = require('lodash');
const baseConfig = require('./base');
const path = require('path');
const webpack = require('webpack');

const config = _.merge(baseConfig, {
  entry: {
    vendor:  Object.keys(require(path.resolve(__dirname, '..', '..', 'package.json')).dependencies),
  },
  output: {
    path: path.resolve(__dirname, '..', '..', '.tmp', 'dll'),
    filename: '[name].dll.js',
    library: '[name]_[hash]'
  },
  cache: false,
  devtool: 'inline-source-map',
  plugins: [
    new webpack.ProvidePlugin({
      $: 'jquery',
      jQuery: 'jquery',
      jquery: 'jquery'
    }),
    new webpack.DefinePlugin({
      __DEV__: true,
      'process.env.NODE_ENV': '"development"'
    }),
    new webpack.DllPlugin({
      path: path.resolve(__dirname, '..', '..', '.tmp', 'dll','[name]-manifest.json'),
      name: "[name]_[hash]",
    }),
  ]
});

config.module.loaders = baseConfig.module.loaders.concat([
  {
    test: /\.css$/,
    loaders: ['style', 'css'],
  },
  {
    test: /\.scss$/,
    loaders: ['style', 'css?sourcemap', 'sass?sourcemap'],
  },
]);

Which is used by my dev.config.js

'use strict';

const config = _.assign(baseConfig, {
 ...
  cache: true,
  devtool: 'cheap-module-inline-source-map',
  plugins: [
    ...,
    new webpack.DllReferencePlugin({
      context: path.resolve(__dirname, '..', '..'),
      manifest: require(path.resolve(__dirname, '..', '..', '.tmp', 'dll', 'vendor-manifest.json')),
    }),
   ...
  ],
});

ianks commented Jun 2, 2016

Here is my dll.config.js file:

'use strict';

const _ = require('lodash');
const baseConfig = require('./base');
const path = require('path');
const webpack = require('webpack');

const config = _.merge(baseConfig, {
  entry: {
    vendor:  Object.keys(require(path.resolve(__dirname, '..', '..', 'package.json')).dependencies),
  },
  output: {
    path: path.resolve(__dirname, '..', '..', '.tmp', 'dll'),
    filename: '[name].dll.js',
    library: '[name]_[hash]'
  },
  cache: false,
  devtool: 'inline-source-map',
  plugins: [
    new webpack.ProvidePlugin({
      $: 'jquery',
      jQuery: 'jquery',
      jquery: 'jquery'
    }),
    new webpack.DefinePlugin({
      __DEV__: true,
      'process.env.NODE_ENV': '"development"'
    }),
    new webpack.DllPlugin({
      path: path.resolve(__dirname, '..', '..', '.tmp', 'dll','[name]-manifest.json'),
      name: "[name]_[hash]",
    }),
  ]
});

config.module.loaders = baseConfig.module.loaders.concat([
  {
    test: /\.css$/,
    loaders: ['style', 'css'],
  },
  {
    test: /\.scss$/,
    loaders: ['style', 'css?sourcemap', 'sass?sourcemap'],
  },
]);

Which is used by my dev.config.js

'use strict';

const config = _.assign(baseConfig, {
 ...
  cache: true,
  devtool: 'cheap-module-inline-source-map',
  plugins: [
    ...,
    new webpack.DllReferencePlugin({
      context: path.resolve(__dirname, '..', '..'),
      manifest: require(path.resolve(__dirname, '..', '..', '.tmp', 'dll', 'vendor-manifest.json')),
    }),
   ...
  ],
});
@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@catamphetamine

catamphetamine Jun 2, 2016

Contributor

gosh, just make that PR, ppl

i can answer any question on this topic

Contributor

catamphetamine commented Jun 2, 2016

gosh, just make that PR, ppl

i can answer any question on this topic

@alecperkey

This comment has been minimized.

Show comment
Hide comment
@alecperkey

alecperkey Jun 2, 2016

@halt-hammerzeit, I'd make the PR would like it working first.

Stuck on this error from my last comment: #616 (comment)

Its some problem with babel react-transform plugin 'not exporting a plugin instance'.
master...alecperkey:master

Any advice?

Looking into it I found https://github.com/gaearon/babel-plugin-react-transform is undergoing deprecation to be replaced by React Hot Loader 3.

alecperkey commented Jun 2, 2016

@halt-hammerzeit, I'd make the PR would like it working first.

Stuck on this error from my last comment: #616 (comment)

Its some problem with babel react-transform plugin 'not exporting a plugin instance'.
master...alecperkey:master

Any advice?

Looking into it I found https://github.com/gaearon/babel-plugin-react-transform is undergoing deprecation to be replaced by React Hot Loader 3.

@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Jun 27, 2016

Contributor

I'm trying to use the DllPlugin, I think I have the Dll files building fine, but when I run my dev server I get:

[1] [webpack-isomorphic-tools] [error] asset not found: ./assets/images/earth-texture.jpg
...
[1] [webpack-isomorphic-tools] [error] asset not found: ./src/components/Columns/Flows.scss
...

It seems like it can't find lots (maybe all of?) my scss and image files to process. Here's my config.dll.js file.

In my webpack/dev.config.js file I include all of the manifests:

 plugins: [
    new webpack.DllReferencePlugin({ 
      context: process.cwd(),
      manifest: require(path.join(assetsPath, 'main-manifest.json'))
    }),
    new webpack.DllReferencePlugin({ 
      context: process.cwd(),
      manifest: require(path.join(assetsPath, 'vendor-manifest.json'))
    }),
    new webpack.DllReferencePlugin({ 
      context: process.cwd(),
      manifest: require(path.join(assetsPath, 'assets-manifest.json'))
    }),

and in main-manifest.json I can see all of the images and scss files listed that webpack-isomorphic-tools are telling me are missing. Do I need to tell isomorphic tools somehow about the manifest json files?

Contributor

AndrewRayCode commented Jun 27, 2016

I'm trying to use the DllPlugin, I think I have the Dll files building fine, but when I run my dev server I get:

[1] [webpack-isomorphic-tools] [error] asset not found: ./assets/images/earth-texture.jpg
...
[1] [webpack-isomorphic-tools] [error] asset not found: ./src/components/Columns/Flows.scss
...

It seems like it can't find lots (maybe all of?) my scss and image files to process. Here's my config.dll.js file.

In my webpack/dev.config.js file I include all of the manifests:

 plugins: [
    new webpack.DllReferencePlugin({ 
      context: process.cwd(),
      manifest: require(path.join(assetsPath, 'main-manifest.json'))
    }),
    new webpack.DllReferencePlugin({ 
      context: process.cwd(),
      manifest: require(path.join(assetsPath, 'vendor-manifest.json'))
    }),
    new webpack.DllReferencePlugin({ 
      context: process.cwd(),
      manifest: require(path.join(assetsPath, 'assets-manifest.json'))
    }),

and in main-manifest.json I can see all of the images and scss files listed that webpack-isomorphic-tools are telling me are missing. Do I need to tell isomorphic tools somehow about the manifest json files?

@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@catamphetamine

catamphetamine Jun 27, 2016

Contributor

@delvarworld I don't think DLL is a place for assets. Conversely, I guess they shouldn't be there.
I don't know why are they there.
Maybe you could try to remove assets from there
https://gist.github.com/DelvarWorld/4294145ba1326b048cd1e17174220ddd#file-dll-config-js-L25

Contributor

catamphetamine commented Jun 27, 2016

@delvarworld I don't think DLL is a place for assets. Conversely, I guess they shouldn't be there.
I don't know why are they there.
Maybe you could try to remove assets from there
https://gist.github.com/DelvarWorld/4294145ba1326b048cd1e17174220ddd#file-dll-config-js-L25

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Jun 27, 2016

Contributor

@halt-hammerzeit part of why I'm doing this is I'm including some very large json files (they are 3d model files) and i'm trying to cache those into separate entry points so they don't all get re-included in the main bundle. But they don't all live in assets.js, in fact none of the errors I'm getting come from that file, they're all from the app entry point ./src/client.js.

I notice the pattern in some places where the image path or css path is required inside the render function - should I be doing this? Right now I require them at the top level of each file like any other import.

Contributor

AndrewRayCode commented Jun 27, 2016

@halt-hammerzeit part of why I'm doing this is I'm including some very large json files (they are 3d model files) and i'm trying to cache those into separate entry points so they don't all get re-included in the main bundle. But they don't all live in assets.js, in fact none of the errors I'm getting come from that file, they're all from the app entry point ./src/client.js.

I notice the pattern in some places where the image path or css path is required inside the render function - should I be doing this? Right now I require them at the top level of each file like any other import.

@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@catamphetamine

catamphetamine Jun 27, 2016

Contributor

@delvarworld Well, I don't know what's going on there but I don't think you can cache assets with webpack-isomorphic-tools (maybe with universal-webpack). If your assets aren't in webpack-stats.json then they won't be found. webpack-stats.json are generated by Webpack.

Contributor

catamphetamine commented Jun 27, 2016

@delvarworld Well, I don't know what's going on there but I don't think you can cache assets with webpack-isomorphic-tools (maybe with universal-webpack). If your assets aren't in webpack-stats.json then they won't be found. webpack-stats.json are generated by Webpack.

@amireh

This comment has been minimized.

Show comment
Hide comment
@amireh

amireh Jun 27, 2016

By the way, posting here since it's relevant to this discussion: there's a pending PR for DLL support (#1201) but it bases off the happypack PR #1090. With some effort, you could define an asset DLL (some README exists for defining more DLLs). There might be issues with webpack-isomorphic-tools though.

Perhaps you could try those patches out?

amireh commented Jun 27, 2016

By the way, posting here since it's relevant to this discussion: there's a pending PR for DLL support (#1201) but it bases off the happypack PR #1090. With some effort, you could define an asset DLL (some README exists for defining more DLLs). There might be issues with webpack-isomorphic-tools though.

Perhaps you could try those patches out?

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Jun 27, 2016

Contributor

@halt-hammerzeit Thanks for the help so far, I've discovered when I run my dll initial build the webpack-assets.json gets populated correctly, or at least it has all the image and scss paths in there. But when I run the default npm run dev it overwrites the contents of that file with:

{
  "javascript": {
    "main": "http://localhost:3001/dist/main-826b5781622f36dec637.js"
  },
  "styles": {},
  "assets": {
    "./~/font-awesome/fonts/fontawesome-webfont.eot": "http://localhost:3001/dist/404a525502f8e5ba7e93b9f02d9e83a9.eot"
  }
}

This is my only change to my wepback/dev.config.js file, other than removing CommonChunksPlugin

+    new webpack.DllReferencePlugin({
+      context: process.cwd(),
+      manifest: require(path.join(assetsPath, 'main-manifest.json'))
+    }),
+    new webpack.DllReferencePlugin({
+      context: process.cwd(),
+      manifest: require(path.join(assetsPath, 'vendor-manifest.json'))
+    }),
+    new webpack.DllReferencePlugin({
+      context: process.cwd(),
+      manifest: require(path.join(assetsPath, 'assets-manifest.json'))
+    }),

For my dev.config.js (not the DLL one), my entry point is simply:

  entry: {
    main: [
      'webpack-hot-middleware/client?path=http://' + host + ':' + port + '/__webpack_hmr',
      'font-awesome-webpack!./src/theme/font-awesome.config.js',
      './src/client.js'
    ],
  },
Contributor

AndrewRayCode commented Jun 27, 2016

@halt-hammerzeit Thanks for the help so far, I've discovered when I run my dll initial build the webpack-assets.json gets populated correctly, or at least it has all the image and scss paths in there. But when I run the default npm run dev it overwrites the contents of that file with:

{
  "javascript": {
    "main": "http://localhost:3001/dist/main-826b5781622f36dec637.js"
  },
  "styles": {},
  "assets": {
    "./~/font-awesome/fonts/fontawesome-webfont.eot": "http://localhost:3001/dist/404a525502f8e5ba7e93b9f02d9e83a9.eot"
  }
}

This is my only change to my wepback/dev.config.js file, other than removing CommonChunksPlugin

+    new webpack.DllReferencePlugin({
+      context: process.cwd(),
+      manifest: require(path.join(assetsPath, 'main-manifest.json'))
+    }),
+    new webpack.DllReferencePlugin({
+      context: process.cwd(),
+      manifest: require(path.join(assetsPath, 'vendor-manifest.json'))
+    }),
+    new webpack.DllReferencePlugin({
+      context: process.cwd(),
+      manifest: require(path.join(assetsPath, 'assets-manifest.json'))
+    }),

For my dev.config.js (not the DLL one), my entry point is simply:

  entry: {
    main: [
      'webpack-hot-middleware/client?path=http://' + host + ':' + port + '/__webpack_hmr',
      'font-awesome-webpack!./src/theme/font-awesome.config.js',
      './src/client.js'
    ],
  },
@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@catamphetamine

catamphetamine Jun 27, 2016

Contributor

@delvarworld

Thanks for the help so far, I've discovered when I run my dll initial build the webpack-assets.json gets populated correctly, or at least it has all the image and scss paths in there. But when I run the default npm run dev it overwrites the contents of that file with.

Oh, maybe that's the issue - it overwrites the file.
You can try to disable webpack-isomorphic-tools plugin for the second pass so that it doesn't overwrite the correst assets file with the new empty one.

Contributor

catamphetamine commented Jun 27, 2016

@delvarworld

Thanks for the help so far, I've discovered when I run my dll initial build the webpack-assets.json gets populated correctly, or at least it has all the image and scss paths in there. But when I run the default npm run dev it overwrites the contents of that file with.

Oh, maybe that's the issue - it overwrites the file.
You can try to disable webpack-isomorphic-tools plugin for the second pass so that it doesn't overwrite the correst assets file with the new empty one.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Jun 27, 2016

Contributor

Ok cool removing webpack-isomorphic-tools from the main webpack config file preserves the manifest. Something else that's confusing is that now all of my assets can only be served from :3000 instead of :3001. I see this line in a linked config, where 1 is subtracted:

publicPath   : 'http://' + host + ':' + ( port - 1 ) + '/dist/'

But I'm not sure why this is needed? When my code starts it logs:

[0] ==> 🚧  Webpack development server listening on port 3001

But all files at :3001 error (they didn't error before the DllPlugin), and those paths only exist at :3000 now. Do you know why this is happening, and if it's an error or expected behavior?

Contributor

AndrewRayCode commented Jun 27, 2016

Ok cool removing webpack-isomorphic-tools from the main webpack config file preserves the manifest. Something else that's confusing is that now all of my assets can only be served from :3000 instead of :3001. I see this line in a linked config, where 1 is subtracted:

publicPath   : 'http://' + host + ':' + ( port - 1 ) + '/dist/'

But I'm not sure why this is needed? When my code starts it logs:

[0] ==> 🚧  Webpack development server listening on port 3001

But all files at :3001 error (they didn't error before the DllPlugin), and those paths only exist at :3000 now. Do you know why this is happening, and if it's an error or expected behavior?

@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@catamphetamine

catamphetamine Jun 27, 2016

Contributor

The files should be available through webpack-dev-server (3001 in your case). If they aren't then it may seem that either their filenames are from another build and therefore are different or that the files aren't included in the (second?) build which could also be true since they are absent from the assets.

Contributor

catamphetamine commented Jun 27, 2016

The files should be available through webpack-dev-server (3001 in your case). If they aren't then it may seem that either their filenames are from another build and therefore are different or that the files aren't included in the (second?) build which could also be true since they are absent from the assets.

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Jun 27, 2016

Contributor

@trueter did you run into this issue and do you have any insight as to why you did port - 1 in your config?

Contributor

AndrewRayCode commented Jun 27, 2016

@trueter did you run into this issue and do you have any insight as to why you did port - 1 in your config?

@AndrewRayCode

This comment has been minimized.

Show comment
Hide comment
@AndrewRayCode

AndrewRayCode Jun 28, 2016

Contributor

Ok, finally got somewhere with the DllPlugin. I'm going to document everything I know here, for now, because the information on this is scattered, hard to find and sometimes convoluted. There are still many unknowns in this process, and I will update this reply if any of the unknowns are clarified.

Warning: It's easy to do this incorrectly. You can actually double your build size by accidentally bundling your dependencies into the Dll bundle and your main bundle. Several people in this thread have said the DllPlugin gave them no speed up, but I wonder if it was just a configuration issue.

Warning: These steps are currently specific to this project, and more specifically to webpack-dev-middleware. However the DllPlugin can be used in any Webpack project.

Warning I haven't set up the final script tag to work in a production build (yet), but it should be straightforward.

TL;DR dll.config.js and dev.config.js and hard code <script src="http://localhost:3001/dist/app.js" charSet="UTF-8"/> into Html.js but you might want to read this anyway!

Reducing My Hot Reload Time From 15s to 4s With The DllPlugin

I still have more work to do on optimizing (next will try HappyPack), but this is a serious improvement already.

My Specific Problem

I have many large asset files that are being built into my bundle. I'm building a 3d game using React and the model files are in JSON, so I have many MB of 3d model data to load. Even though they're going through the file loader, they still get built into the main bundle. I'm also including large Javascript libraries, like Three.js.

Webpack Is Slow

Out of the box, Webpack isn't efficient. In fact, it's surprising it works the way it does. Webpack treats everything, both your source and your dependencies, as one big bundle. Even in development mode. What this means in practice is that if you change any source file, ALL Javascript files, including dependencies, are re-processed and re-bundled. You can see why this would be slow. I believe this happens even if you use the CommonChunks plugin. If you change your source file, even with CommonChunks, the common chunk will get re-written (I could be wrong but that's what the last section of this post suggests).

What Is The DllPlugin?

Really, if you think about it, we don't want Webpack to touch our third party dependencies more than once. That's where the oddly named DllPlugin comes in. I don't know why it reuses the Windows development "Dynamically Linked Library" terminology, but that's what it stands for. The basic workflow is this:

  1. Create a separate, one time build step that uses the DllPlugin
  2. Run this one-time build step before your dev server. It creates both a bundle file containing your third party code, and a "manifest" file, which maps require paths to built Webpack ids, so your app code (via Webpack) knows how to find them at runtime.
  3. Modify your webpack dev config to use the DllReferencePlugin to point to the manifest file(s) built in step 2.

Don't overlook the importance of caching this manifest file. Without the DllPlugin it's re-created every time for every code change, as in Webpack walks all your dependencies to build it. Caching it for third party node modules seems like a good idea to me!

Setting up the DllPlugin

You have two options here. You can either create an entirely separate Webpack config file for the Dll build, or you can modify your existing dev.config.js file to build the Dll files if you set an environment variable. The benefit of an environment variable switch is less code duplication, the downside is it's harder to read and makes the config file more confusing. The downside of a separate file is you'll probably duplicate some config options, violating the DRY principle. I don't think either option is ideal, so go with personal preference. For this tutorial, I'm going to create a separate dll.config.js file.

Creating The Dll Configuration File

This will be a modified copy of your dev.config.js file. We'll go over each thing you need to change.

I've put the my completed dll.config.js file online for you to reference.

Every step below is done in a copy of webpack/dev.config.js that I arbitrarily name webpack/dll.config.js

1) Add an entry for your main app, but name it something different. Also it must be an array, not a single string.

entry: {
  app_assets: [ './src/client.js' ],

According to this comment, if you don't include your main entry file, the main webpack-assets.json manifest file won't include your runtime static assets like CSS, JS, etc.

2) Add an array of the third party vendor code to go into the cached manifest and bundle:

entry: {
  app_assets: [ './src/client.js' ],
  vendor: [
    'react',
    'react-dom',
    ...
  ]

How did I come up with this list? Based on an earlier comment, I modified webpack-dev-server.js to set quiet: false and noInfo: false, started the dev server, then manually copied anything starting with ./~/ into the vendor: [] array. Well, most things. I excluded Webpack-looking things like the css-loader which appears in that list.

One gotcha here is if you try to include 'babel-runtime' (which is a huge dependency) in the array it will fail, complaining it can't find it. I don't know why, but change it to 'babel-runtime/core-js'.

By the way, when this is all over, you can set the server flags again (quiet: false and noInfo: false,) and see which files are now coming from your cached manifest. They will look like:

delegated ./node_modules/redux/lib/index.js from dll-reference vendor 42 bytes {0} [not cacheable]

3) Set your output section to exactly:

  output: {
    path: assetsPath,
    filename: '[name].dll.js',
    library: '[name]',
    publicPath: 'http://' + host + ':' + ( port - 1 ) + '/dist/'
  },

Through trial and error I found that library: '[name]', is required. If you want you can change the .dll.js to something, like .bundle.js, but keep [name].

4) Include the DllPlugin

    new webpack.DllPlugin({
      name: '[name]',
      path: path.join( assetsPath, '[name]-manifest.json' ),
    }),

You may have noticed that earlier comments on this thread include things like context: undefined and other config keys. You don't need them. Again you can change the -manifest.json string, but remember what you change it to because we need it.

5) Verify your plugins

Make sure you include webpackIsomorphicToolsPlugin.development(), in the plugins: [] list for your dll.config.js file.

Modifying Your Main dev.config.js file

My final dev.config.js file is online for you to compare against.

1) Remove everything from your entry config except your main entry point

  entry: {
    main: [
      'webpack-hot-middleware/client?path=http://' + host + ':' + port + '/__webpack_hmr',
      'font-awesome-webpack!./src/theme/font-awesome.config.js',
      './src/client.js'
    ],
  },

Standard stuff here.

2) Modify your output to only build one file with an exact name

  output: {
    path: assetsPath,
    filename: 'app.js',
    publicPath: 'http://' + host + ':' + port + '/dist/'
  },

We'll come back to this later, see Unknown 2 at the end of this post for why we don't use the good old [name]-[hash] style.

3) Add DllReferencePlugin(s) to your plugins list:

  plugins: [
    new webpack.DllReferencePlugin({ 
      context: path.join( __dirname, '../' ),
      manifest: require(path.join(assetsPath, 'vendor-manifest.json')),
    }),
    ....

Add one DllReferencePlugin call for each entry you specify in dll.config.js. If you only have one vendor entry (most common case) then you only need one plugin here. Keep in mind it must match the string pattern you specified in dll.config.js (in our case [name]-manifest.json).

Do they need to go at the beginning of the plugins list? I don't know. A previous comment used unshift to put the DllReferencePlugin at the beginning of the array. I'm not deviating.

4) Remove the webpackIsomorphicToolsPlugin and CommonsChunkPlugin

Do not include webpackIsomorphicToolsPlugin nor CommonsChunkPlugin in your plugins: [] list for webpack/dev.config.js. CommonChunks can't co-exist with DllPlugin. They are competing solutions, and one clobbers the other. For why we remove webpackIsomorphicToolsPlugin, see Unknown 2

Add your new bundles to Html.js

In this project's Html.js file, include your vendor script tag(s):

<script src={assets.javascript.vendor} charSet="UTF-8"/>

Now include your hard coded app.js file, which we created in the above dev.config.js file:

<script src="http://localhost:3001/dist/app.js" charSet="UTF-8"/>

Run the manifest step and then run your project!

Here's the command I use to create the cached manifest/bundle files. You only have to run this command once, and then only again if your dependencies change:

webpack --colors --progress --config webpack/dll.config.js

Then start your server like normal:

npm run dev

If everything went well, you should have faster hot reloading!

Unknowns

Unknown 1:

According to @sokra:

If you process the file with the file-loader, they should not be included in the bundle file, but emitted as separate files

However, this boilerplate project uses webpack-isomorphic-tools and webpack-dev-middleware instead of the regular dev server, and according to the middleware docs:

No files are written to disk, it handle the files in memory

Does using webpack-dev-middleware mean that large files are always re-created in memory on every hot reload? Would this be solved by using the regular dev server and just putting files on disk that aren't touched again on hot reload? I have no idea, but I made a ticket to ask.

Unknown 2:

In the above code we removed the isomorphic tools plugin from the main dev config, and hard coded the asset path. So here's the issue. If I include webpackIsomorphicToolsPlugin in my dev.config.js file, it overwrites the contents of webpack-assets.json, which was previously created by the Dll step. This means that all the cached manifest data is clobbered. I have no idea if this is supposed to work this way, and if you're not supposed to use it in the main dev server, but...

...The other problem here is that without webpackIsomorphicToolsPlugin running on your source code, assets.javascript.main won't exist, because it's not added to the manifest. I believe webpackIsomorphicToolsPlugin is reponsible for the addition. That's why I have to hard code the path. Also this means if you want to load it from the same port as the other assets (which are 3000) you'd have to manually expose that file by adding a server route.

That's it!

This is a first draft, and hopefully it gets some good input and feedback and is refined. I don't know if this is the solution, or even an ideal solution. I've just been hacking at this frustrating problem for 2 days straight and had to jump between many sources. The Webpack docs are no help, as expected. Hopefully this helps someone!

Further reading: Webpack Plugins we been keepin on the DLL and Optimizing Webpack Build Times

(Spam) if this helped you consider following me on Twitter

Contributor

AndrewRayCode commented Jun 28, 2016

Ok, finally got somewhere with the DllPlugin. I'm going to document everything I know here, for now, because the information on this is scattered, hard to find and sometimes convoluted. There are still many unknowns in this process, and I will update this reply if any of the unknowns are clarified.

Warning: It's easy to do this incorrectly. You can actually double your build size by accidentally bundling your dependencies into the Dll bundle and your main bundle. Several people in this thread have said the DllPlugin gave them no speed up, but I wonder if it was just a configuration issue.

Warning: These steps are currently specific to this project, and more specifically to webpack-dev-middleware. However the DllPlugin can be used in any Webpack project.

Warning I haven't set up the final script tag to work in a production build (yet), but it should be straightforward.

TL;DR dll.config.js and dev.config.js and hard code <script src="http://localhost:3001/dist/app.js" charSet="UTF-8"/> into Html.js but you might want to read this anyway!

Reducing My Hot Reload Time From 15s to 4s With The DllPlugin

I still have more work to do on optimizing (next will try HappyPack), but this is a serious improvement already.

My Specific Problem

I have many large asset files that are being built into my bundle. I'm building a 3d game using React and the model files are in JSON, so I have many MB of 3d model data to load. Even though they're going through the file loader, they still get built into the main bundle. I'm also including large Javascript libraries, like Three.js.

Webpack Is Slow

Out of the box, Webpack isn't efficient. In fact, it's surprising it works the way it does. Webpack treats everything, both your source and your dependencies, as one big bundle. Even in development mode. What this means in practice is that if you change any source file, ALL Javascript files, including dependencies, are re-processed and re-bundled. You can see why this would be slow. I believe this happens even if you use the CommonChunks plugin. If you change your source file, even with CommonChunks, the common chunk will get re-written (I could be wrong but that's what the last section of this post suggests).

What Is The DllPlugin?

Really, if you think about it, we don't want Webpack to touch our third party dependencies more than once. That's where the oddly named DllPlugin comes in. I don't know why it reuses the Windows development "Dynamically Linked Library" terminology, but that's what it stands for. The basic workflow is this:

  1. Create a separate, one time build step that uses the DllPlugin
  2. Run this one-time build step before your dev server. It creates both a bundle file containing your third party code, and a "manifest" file, which maps require paths to built Webpack ids, so your app code (via Webpack) knows how to find them at runtime.
  3. Modify your webpack dev config to use the DllReferencePlugin to point to the manifest file(s) built in step 2.

Don't overlook the importance of caching this manifest file. Without the DllPlugin it's re-created every time for every code change, as in Webpack walks all your dependencies to build it. Caching it for third party node modules seems like a good idea to me!

Setting up the DllPlugin

You have two options here. You can either create an entirely separate Webpack config file for the Dll build, or you can modify your existing dev.config.js file to build the Dll files if you set an environment variable. The benefit of an environment variable switch is less code duplication, the downside is it's harder to read and makes the config file more confusing. The downside of a separate file is you'll probably duplicate some config options, violating the DRY principle. I don't think either option is ideal, so go with personal preference. For this tutorial, I'm going to create a separate dll.config.js file.

Creating The Dll Configuration File

This will be a modified copy of your dev.config.js file. We'll go over each thing you need to change.

I've put the my completed dll.config.js file online for you to reference.

Every step below is done in a copy of webpack/dev.config.js that I arbitrarily name webpack/dll.config.js

1) Add an entry for your main app, but name it something different. Also it must be an array, not a single string.

entry: {
  app_assets: [ './src/client.js' ],

According to this comment, if you don't include your main entry file, the main webpack-assets.json manifest file won't include your runtime static assets like CSS, JS, etc.

2) Add an array of the third party vendor code to go into the cached manifest and bundle:

entry: {
  app_assets: [ './src/client.js' ],
  vendor: [
    'react',
    'react-dom',
    ...
  ]

How did I come up with this list? Based on an earlier comment, I modified webpack-dev-server.js to set quiet: false and noInfo: false, started the dev server, then manually copied anything starting with ./~/ into the vendor: [] array. Well, most things. I excluded Webpack-looking things like the css-loader which appears in that list.

One gotcha here is if you try to include 'babel-runtime' (which is a huge dependency) in the array it will fail, complaining it can't find it. I don't know why, but change it to 'babel-runtime/core-js'.

By the way, when this is all over, you can set the server flags again (quiet: false and noInfo: false,) and see which files are now coming from your cached manifest. They will look like:

delegated ./node_modules/redux/lib/index.js from dll-reference vendor 42 bytes {0} [not cacheable]

3) Set your output section to exactly:

  output: {
    path: assetsPath,
    filename: '[name].dll.js',
    library: '[name]',
    publicPath: 'http://' + host + ':' + ( port - 1 ) + '/dist/'
  },

Through trial and error I found that library: '[name]', is required. If you want you can change the .dll.js to something, like .bundle.js, but keep [name].

4) Include the DllPlugin

    new webpack.DllPlugin({
      name: '[name]',
      path: path.join( assetsPath, '[name]-manifest.json' ),
    }),

You may have noticed that earlier comments on this thread include things like context: undefined and other config keys. You don't need them. Again you can change the -manifest.json string, but remember what you change it to because we need it.

5) Verify your plugins

Make sure you include webpackIsomorphicToolsPlugin.development(), in the plugins: [] list for your dll.config.js file.

Modifying Your Main dev.config.js file

My final dev.config.js file is online for you to compare against.

1) Remove everything from your entry config except your main entry point

  entry: {
    main: [
      'webpack-hot-middleware/client?path=http://' + host + ':' + port + '/__webpack_hmr',
      'font-awesome-webpack!./src/theme/font-awesome.config.js',
      './src/client.js'
    ],
  },

Standard stuff here.

2) Modify your output to only build one file with an exact name

  output: {
    path: assetsPath,
    filename: 'app.js',
    publicPath: 'http://' + host + ':' + port + '/dist/'
  },

We'll come back to this later, see Unknown 2 at the end of this post for why we don't use the good old [name]-[hash] style.

3) Add DllReferencePlugin(s) to your plugins list:

  plugins: [
    new webpack.DllReferencePlugin({ 
      context: path.join( __dirname, '../' ),
      manifest: require(path.join(assetsPath, 'vendor-manifest.json')),
    }),
    ....

Add one DllReferencePlugin call for each entry you specify in dll.config.js. If you only have one vendor entry (most common case) then you only need one plugin here. Keep in mind it must match the string pattern you specified in dll.config.js (in our case [name]-manifest.json).

Do they need to go at the beginning of the plugins list? I don't know. A previous comment used unshift to put the DllReferencePlugin at the beginning of the array. I'm not deviating.

4) Remove the webpackIsomorphicToolsPlugin and CommonsChunkPlugin

Do not include webpackIsomorphicToolsPlugin nor CommonsChunkPlugin in your plugins: [] list for webpack/dev.config.js. CommonChunks can't co-exist with DllPlugin. They are competing solutions, and one clobbers the other. For why we remove webpackIsomorphicToolsPlugin, see Unknown 2

Add your new bundles to Html.js

In this project's Html.js file, include your vendor script tag(s):

<script src={assets.javascript.vendor} charSet="UTF-8"/>

Now include your hard coded app.js file, which we created in the above dev.config.js file:

<script src="http://localhost:3001/dist/app.js" charSet="UTF-8"/>

Run the manifest step and then run your project!

Here's the command I use to create the cached manifest/bundle files. You only have to run this command once, and then only again if your dependencies change:

webpack --colors --progress --config webpack/dll.config.js

Then start your server like normal:

npm run dev

If everything went well, you should have faster hot reloading!

Unknowns

Unknown 1:

According to @sokra:

If you process the file with the file-loader, they should not be included in the bundle file, but emitted as separate files

However, this boilerplate project uses webpack-isomorphic-tools and webpack-dev-middleware instead of the regular dev server, and according to the middleware docs:

No files are written to disk, it handle the files in memory

Does using webpack-dev-middleware mean that large files are always re-created in memory on every hot reload? Would this be solved by using the regular dev server and just putting files on disk that aren't touched again on hot reload? I have no idea, but I made a ticket to ask.

Unknown 2:

In the above code we removed the isomorphic tools plugin from the main dev config, and hard coded the asset path. So here's the issue. If I include webpackIsomorphicToolsPlugin in my dev.config.js file, it overwrites the contents of webpack-assets.json, which was previously created by the Dll step. This means that all the cached manifest data is clobbered. I have no idea if this is supposed to work this way, and if you're not supposed to use it in the main dev server, but...

...The other problem here is that without webpackIsomorphicToolsPlugin running on your source code, assets.javascript.main won't exist, because it's not added to the manifest. I believe webpackIsomorphicToolsPlugin is reponsible for the addition. That's why I have to hard code the path. Also this means if you want to load it from the same port as the other assets (which are 3000) you'd have to manually expose that file by adding a server route.

That's it!

This is a first draft, and hopefully it gets some good input and feedback and is refined. I don't know if this is the solution, or even an ideal solution. I've just been hacking at this frustrating problem for 2 days straight and had to jump between many sources. The Webpack docs are no help, as expected. Hopefully this helps someone!

Further reading: Webpack Plugins we been keepin on the DLL and Optimizing Webpack Build Times

(Spam) if this helped you consider following me on Twitter

@catamphetamine

This comment has been minimized.

Show comment
Hide comment
@catamphetamine

catamphetamine Jun 28, 2016

Contributor

Good post

Does using webpack-dev-middleware mean that large files are always re-created in memory on every hot reload?

If your file is processed by webpack dev server (in any way) then it resides in RAM and is recompiled only on changes to it. I would suppose the DDLRefPlugin is intelligent enough to exclude files from the DLL from webpack dev server's list of files.

I have no idea if this is supposed to work this way

It is supposed to always overwrite webpack-assets.json on a subsequent run.

and if you're not supposed to use it in the main dev server, but...

You do it in the main dev server if you're not gonna put your assets into the DLL. Otherwise, I guess, you do it in the DLL compilation run (for whatever reasons you have).

without webpackIsomorphicToolsPlugin running on your source code, assets.javascript.main won't exist

Yes, assets.javascript.main is taken from webpack-assets.json, so if it's wrong or non-existent there then it will be the same when you run the app.

(you might also consider trying universal-webpack later)

Contributor

catamphetamine commented Jun 28, 2016

Good post

Does using webpack-dev-middleware mean that large files are always re-created in memory on every hot reload?

If your file is processed by webpack dev server (in any way) then it resides in RAM and is recompiled only on changes to it. I would suppose the DDLRefPlugin is intelligent enough to exclude files from the DLL from webpack dev server's list of files.

I have no idea if this is supposed to work this way

It is supposed to always overwrite webpack-assets.json on a subsequent run.

and if you're not supposed to use it in the main dev server, but...

You do it in the main dev server if you're not gonna put your assets into the DLL. Otherwise, I guess, you do it in the DLL compilation run (for whatever reasons you have).

without webpackIsomorphicToolsPlugin running on your source code, assets.javascript.main won't exist

Yes, assets.javascript.main is taken from webpack-assets.json, so if it's wrong or non-existent there then it will be the same when you run the app.

(you might also consider trying universal-webpack later)

@amireh

This comment has been minimized.

Show comment
Hide comment
@amireh

amireh Jun 28, 2016

The solution @delvarworld outlined in great detail is close to the one proposed in PR (#1201), perhaps these should converge? The only differences as far as I can tell is that it doesn't disable CommonsChunk or isomorphic-tools (although it probably should) and it tracks babel-runtime modules in the vendor DLL.

Edit: sound less obnoxious - was just trying to help. 😁 Have a great week everyone!

amireh commented Jun 28, 2016

The solution @delvarworld outlined in great detail is close to the one proposed in PR (#1201), perhaps these should converge? The only differences as far as I can tell is that it doesn't disable CommonsChunk or isomorphic-tools (although it probably should) and it tracks babel-runtime modules in the vendor DLL.

Edit: sound less obnoxious - was just trying to help. 😁 Have a great week everyone!

@Jack-Barry

This comment has been minimized.

Show comment
Hide comment
@Jack-Barry

Jack-Barry Aug 4, 2016

It's not all that elegant but what I've done to make hot updates snappy is to run a separate, concurrent Webpack process to bundle my Bootstrap, with a configuration that doesn't have hot updates enabled. In simple terms, the basic syntax is:

module.exports = [
    { config for regular bundle with hot updates enabled, css loader ignores bootstrap },
    { config for Bootstrap using raw loader without hot updates }
];

This way, it gets passed through the loaders on initial build, but isn't touched during incremental builds. It's also running concurrently on the initial build with my regular bundle which shaves a few seconds off at that point. Still pretty long build time initially, but it seems to help for incremental builds which are more aggravating when slow. I'd still like to speed up that initial build but that seems a lot more doable using something like this. I'm thinking of setting up a webpack-vendor.config.js so I can do something like run webpack on that as needed (just needs to build the files once on a new machine, or be re-run whenever dependencies are updated), and use webpack-dev-server with hot updates to serve up my actual app code, but I need sleep first.

I'm no pro so if this approach would bite me in the ass please let me know before I dig too deep.

Jack-Barry commented Aug 4, 2016

It's not all that elegant but what I've done to make hot updates snappy is to run a separate, concurrent Webpack process to bundle my Bootstrap, with a configuration that doesn't have hot updates enabled. In simple terms, the basic syntax is:

module.exports = [
    { config for regular bundle with hot updates enabled, css loader ignores bootstrap },
    { config for Bootstrap using raw loader without hot updates }
];

This way, it gets passed through the loaders on initial build, but isn't touched during incremental builds. It's also running concurrently on the initial build with my regular bundle which shaves a few seconds off at that point. Still pretty long build time initially, but it seems to help for incremental builds which are more aggravating when slow. I'd still like to speed up that initial build but that seems a lot more doable using something like this. I'm thinking of setting up a webpack-vendor.config.js so I can do something like run webpack on that as needed (just needs to build the files once on a new machine, or be re-run whenever dependencies are updated), and use webpack-dev-server with hot updates to serve up my actual app code, but I need sleep first.

I'm no pro so if this approach would bite me in the ass please let me know before I dig too deep.

@seeliang

This comment has been minimized.

Show comment
Hide comment
@seeliang

seeliang Sep 30, 2016

Thanks @trueter,
with the babel cacheDirectory set
my rebuild drops form 2.5s to 1.5s

seeliang commented Sep 30, 2016

Thanks @trueter,
with the babel cacheDirectory set
my rebuild drops form 2.5s to 1.5s

@seeliang

This comment has been minimized.

Show comment
Hide comment
@seeliang

seeliang Oct 5, 2016

Try cache : true,

Now 1.5s to build 0.6s to rebuild !!

module.exports = {
  cache: true,
  devtool: 'cheap-module-eval-source-map',
  entry: './src/scripts/index.js',
  externals: {

  },

  module: {
    loaders: [
      {
        loader: 'babel-loader',
        exclude: /node_modules/,
        query: {
          presets: ['es2015'],
          cacheDirectory: true
        }
      }
    ]
  },

  output: {

  },

  plugins: [
    new webpack.optimize.UglifyJsPlugin({ output: {comments: false}})
  ],

  resolve: {
    modulesDirectories: [
      'node_modules'
    ]
  }
}`

seeliang commented Oct 5, 2016

Try cache : true,

Now 1.5s to build 0.6s to rebuild !!

module.exports = {
  cache: true,
  devtool: 'cheap-module-eval-source-map',
  entry: './src/scripts/index.js',
  externals: {

  },

  module: {
    loaders: [
      {
        loader: 'babel-loader',
        exclude: /node_modules/,
        query: {
          presets: ['es2015'],
          cacheDirectory: true
        }
      }
    ]
  },

  output: {

  },

  plugins: [
    new webpack.optimize.UglifyJsPlugin({ output: {comments: false}})
  ],

  resolve: {
    modulesDirectories: [
      'node_modules'
    ]
  }
}`
@echenley

This comment has been minimized.

Show comment
Hide comment
@echenley

echenley Oct 6, 2016

@seeliang cache: true is already enabled by default in watch mode https://webpack.github.io/docs/configuration.html#cache. I've noticed with Webpack 1.x the first rebuild takes longer than subsequent rebuilds, which can be a red herring when tweaking your build.

echenley commented Oct 6, 2016

@seeliang cache: true is already enabled by default in watch mode https://webpack.github.io/docs/configuration.html#cache. I've noticed with Webpack 1.x the first rebuild takes longer than subsequent rebuilds, which can be a red herring when tweaking your build.

@seeliang

This comment has been minimized.

Show comment
Hide comment
@seeliang

seeliang Oct 7, 2016

@echenley thanks for the update.

I really not 100% sure why i need cache: true or how it works,
one post mentioned this set

and it did improve my build :)

seeliang commented Oct 7, 2016

@echenley thanks for the update.

I really not 100% sure why i need cache: true or how it works,
one post mentioned this set

and it did improve my build :)

@seeliang

This comment has been minimized.

Show comment
Hide comment
@seeliang

seeliang commented Oct 12, 2016

if you are using babel with lodash

try
https://github.com/lodash/babel-plugin-lodash

@davidnguyen179

This comment has been minimized.

Show comment
Hide comment
@davidnguyen179

davidnguyen179 Oct 15, 2016

I am using Happypack & DLL plugin together. Currently, the performance is not so bad. It reduced from more than 15s to 4-5s when it rebuild

davidnguyen179 commented Oct 15, 2016

I am using Happypack & DLL plugin together. Currently, the performance is not so bad. It reduced from more than 15s to 4-5s when it rebuild

@vhpoet

This comment has been minimized.

Show comment
Hide comment
@vhpoet

vhpoet Oct 18, 2016

has anyone tried webpack unsafe caching (another link)? This is also supposed to make incremental builds faster.

vhpoet commented Oct 18, 2016

has anyone tried webpack unsafe caching (another link)? This is also supposed to make incremental builds faster.

@bebraw

This comment has been minimized.

Show comment
Hide comment
@bebraw

bebraw Dec 6, 2016

hard-source-webpack-plugin might come in handy here.

bebraw commented Dec 6, 2016

hard-source-webpack-plugin might come in handy here.

@nezed

This comment has been minimized.

Show comment
Hide comment
@nezed

nezed Dec 14, 2016

Also you can easily improve performance by moving modules that you have no intention of changing into new chunks by specific/extra conditions (e.g. module path or module size) 🔥 🚀

Check the webpack-split-chunks

Example:

    plugins: [
        new ChunksPlugin({
            to: 'vendor',
            test: /node_modules/ // can be function | RegExp | Array[RegExp] 
        })
    ]

nezed commented Dec 14, 2016

Also you can easily improve performance by moving modules that you have no intention of changing into new chunks by specific/extra conditions (e.g. module path or module size) 🔥 🚀

Check the webpack-split-chunks

Example:

    plugins: [
        new ChunksPlugin({
            to: 'vendor',
            test: /node_modules/ // can be function | RegExp | Array[RegExp] 
        })
    ]
@oychao

This comment has been minimized.

Show comment
Hide comment
@oychao

oychao Jan 19, 2017

@bdefore Many thanks. You made my day.

oychao commented Jan 19, 2017

@bdefore Many thanks. You made my day.

kvz added a commit to kvz/lanyon that referenced this issue Mar 23, 2017

@Pines-Cheng

This comment has been minimized.

Show comment
Hide comment
@Pines-Cheng

Pines-Cheng Jun 26, 2017

my rebuild time is 12s,what's wrong with my project...

Pines-Cheng commented Jun 26, 2017

my rebuild time is 12s,what's wrong with my project...

@Pines-Cheng

This comment has been minimized.

Show comment
Hide comment
@Pines-Cheng

Pines-Cheng Jun 27, 2017

after used dll plugin,my project build time is less than 30s,but the rebuild time is still 13s... WTF

Pines-Cheng commented Jun 27, 2017

after used dll plugin,my project build time is less than 30s,but the rebuild time is still 13s... WTF

@seeliang

This comment has been minimized.

Show comment
Hide comment
@seeliang

seeliang Jun 28, 2017

@Pines-Cheng To optimise rebuild, i think you can start with separating dev build and prod build, try move vendor js files out of build. that is what i can think of for now

seeliang commented Jun 28, 2017

@Pines-Cheng To optimise rebuild, i think you can start with separating dev build and prod build, try move vendor js files out of build. that is what i can think of for now

@up209d

This comment has been minimized.

Show comment
Hide comment
@up209d

up209d Apr 5, 2018

Ran into the slow performance today when I cannot afford the snail speed of rebuilding.

My case was 8s first build (with DLL) and another 8-10s for each rebuild (WTH???). It turned out that webpack was not a guy to blame but babel (babel-loader). Tried to work out with happypack but got no gain.

Well, in the end, it is not actually babel but a babel plugin babel-plugin-styled-components as I am using styled-components stuffs atm. Ver 1.5.0 was like adding 5s to every transformation. The problem was gone when I updated to ver 1.5.1. So now my rebuilds only take 1.5 - 2s.

So my lesson is that, in general, if you get some performance issue with the first build, webpack is most likely the issue origin. If you get speed issue with the re-build, loaders like (babel-loader,sass-loader) are the ones need to be investigated at first. Make sure that you checked all plugins you added to babel transformation, there is a high chance that you will found a solution there.

up209d commented Apr 5, 2018

Ran into the slow performance today when I cannot afford the snail speed of rebuilding.

My case was 8s first build (with DLL) and another 8-10s for each rebuild (WTH???). It turned out that webpack was not a guy to blame but babel (babel-loader). Tried to work out with happypack but got no gain.

Well, in the end, it is not actually babel but a babel plugin babel-plugin-styled-components as I am using styled-components stuffs atm. Ver 1.5.0 was like adding 5s to every transformation. The problem was gone when I updated to ver 1.5.1. So now my rebuilds only take 1.5 - 2s.

So my lesson is that, in general, if you get some performance issue with the first build, webpack is most likely the issue origin. If you get speed issue with the re-build, loaders like (babel-loader,sass-loader) are the ones need to be investigated at first. Make sure that you checked all plugins you added to babel transformation, there is a high chance that you will found a solution there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment