Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally disable dynamic requires #198

Closed
hunterloftis opened this issue Mar 11, 2014 · 33 comments

Comments

Projects
None yet
@hunterloftis
Copy link

commented Mar 11, 2014

This 1-line file, when built with webpack, becomes 128K:

var moment = require('moment');

It seems that webpack is tripping up over the dynamic language requires and including all 60+ languages that come with moment. I'd love a mechanism of turning off this feature that can sometimes be too smart for its own good.

@jmorrel @petehunt this was the culprit behind the huge file sizes.

relevant: #59

@sokra would you accept a patch enabling something like resolve.dynamic: false?

@jhnns

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

I'm afraid resolve.dynamic: false won't help, because dynamic requires are unexecutable in the browser (unless, every file in the given directory is included).

I had the same issue with moment.js. Check out the discussion.

We could also do a pull-request to moment.js, because I don't think that dynamic requires are really necessary.

@jhnns

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

Oops, sorry, I skipped that you already know about the ContextReplacementPlugin. Doesn't it help?

sokra added a commit that referenced this issue Mar 11, 2014

allow to configure default RegExps for automatically created contexts
allow to configure when an automatically created context is critical
better warning message in critical dependencies warning

fixes #196 #198

@sokra sokra closed this Mar 11, 2014

sokra added a commit that referenced this issue Mar 11, 2014

allow to configure default RegExps for automatically created contexts
allow to configure when an automatically created context is critical
better warning message in critical dependencies warning

fixes #196
fixes #198
@sokra

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

I added options to change the default RegExps for contexts. This can be used to disable dynamic requires:

{
  module: {
    // require
    unknownContextRegExp: /$^/,
    unknownContextCritical: false,

    // require(expr)
    exprContextRegExp: /$^/,
    exprContextCritical: false,

    // require("prefix" + expr + "surfix")
    wrappedContextRegExp: /$^/,
    wrappedContextCritical: false
  }
}
@sokra

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

To limit the languages for moment: #59 (comment)

@hunterloftis

This comment has been minimized.

Copy link
Author

commented Mar 11, 2014

I wonder if there's a simpler way to enable this? I wouldn't have known something was wrong if I hadn't just come from browserify or if the problem were small... a few languages instead of 60. Browserify builds into 164k while webpack builds into 300k.

Adding the above module properties doesn't change anything, while adding the moment-specific context plugin brings that down to 200k.

I'd love to find a general-purpose solution that would enable webpack to output files closer to browserify's size without library-specific plugin fixes.

@sokra

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

webpack just includes what the module requires. moment requires all its languages so they are included in the bundle. Just because browserify doesn't "find" that dependencies, we don't have to adopt this (wrong) behavior. The defaults are choosen to get the best support for modules. If we don't include the languages moment won't work or you would need to manually include the languages.

You can change the default with the ContentReplacementPlugin for a specified directory. This is only a one line addition and you need to choose the languages anyway.

browserify:

  • just require("moment") won't work, because no languages are loaded.
  • choose the languages by require("moment") plus require("moment/lang/en")

webpack:

  • just require("moment") will work. all languages are included.
  • choose the languages with the ContextReplacementPlugin and do require("moment")
@hunterloftis

This comment has been minimized.

Copy link
Author

commented Mar 11, 2014

Weird, I know that browserify require('moment') does work (because it's working right now with our codebase). It's just using the default English, I suppose.

I'd ideally like to disable all of the smart/regex matching for require. I couldn't find any docs for unknownContextRegExp et al but maybe I can use ContextReplacementPlugin to match nothing.

@jhnns

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

Maybe they have a switch in their code that skips the dynamic require in browsers. Strange why they still have it in node... dynamic requires are an anti-pattern anyway.

@sokra

This comment has been minimized.

Copy link
Member

commented Mar 11, 2014

English is embedded in the main file. The dynamic require is wrapped in a try catch block.

@jhnns

This comment has been minimized.

Copy link
Member

commented Mar 12, 2014

That's why browserify "just" works ^^.

But I have to admit that bundling all files in the directory without a warning isn't the best way to go either. What about a warning about "implicit contexts" that aren't constrained by a regexp? This should be a weak warning of course. Could it be printed as optimization tip or something?

@hunterloftis

This comment has been minimized.

Copy link
Author

commented Mar 12, 2014

I ended up using this and it works quite well:

new webpack.ContextReplacementPlugin(/.*$/, /a^/)

Any tips on quieting the output from webpack? Couldn't find that in the docs but I'd love to not get lists and lists of stuff whenever it's run.

@kflorence

This comment has been minimized.

Copy link

commented Jul 24, 2014

Just ran into this issue as well. I am confused about how to use the ContextReplacementPlugin, and reading the docs page doesn't help much. The initial suggestion by @sokra on #87 kind of makes sense, but I don't understand your final solution @hunterloftis

What If I don't want any language files to load? with the above solution I see:

   [56] ./~/moment/moment.js 84976 {6} [optional] [built]
   [57] ./~/moment/lang a^ 160 {6} [optional] [built]

Which tells me the a^ is still adding 160 bytes?

@taurose

This comment has been minimized.

Copy link

commented Jul 24, 2014

I think hunterloftis code effectively excludes any files from contexts / dynamic requires. It tells webpack that whenever it finds a require context like that of moment to not include any file at all. First regex matches every context, second argument (new context) matches nothing.
I guess the better (more specific) solution is to do this:

 new webpack.ContextReplacementPlugin(/moment[\/\\]lang$/, /a^/),

The extra bytes are boilerplate code for resolving dynamic requires, ie it maps the paths of the modules that were included to their module ids, and webpack-require's them as requested, or throws an error.
I guess this could be optimized to just always throw an error if no module is included.

This also works for me, without any overhead (error is thrown directly inside momentjs module):

new webpack.IgnorePlugin(/^\.\/lang$/)

There doesn't seem to be a way to restrict that to momentjs though, so I think this would cause problems when you use contexts to load from a "lang" folder elsewhere.
Not sure if there's a clean way to avoid that, but here's a custom plugin that can solve that (copied from IgnorePlugin):

            function () {
                this.plugin("context-module-factory", function(cmf) {
                    cmf.plugin("before-resolve", function(result, callback) {
                        if(!result) return callback();
                        if(result.context === path.join(__dirname, 'node_modules', 'moment')  // You might have to customize that path
                            && result.request === './lang') {
                            return callback();
                        }
                        return callback(null, result);
                    });
                });
            }
@kflorence

This comment has been minimized.

Copy link

commented Jul 25, 2014

@taurose Thanks for the in-depth explanation. I guess my original confusion arose from the fact that I didn't know what was meant by "context" -- but it seems that it just means the path at which a require is being made, similar to the "context" option for entries? Does it only apply to dynamic includes?

Anyways, I think I understand now, but I also think there should be an easier and more intuitive way of accomplishing this.

@taurose

This comment has been minimized.

Copy link

commented Jul 26, 2014

I'm not too familiar with the terminology and internals of webpack either. There is some documentation about contexts here: https://webpack.github.io/docs/context.html . It also explains how to use require.context to create your own contexts.

My understanding is that every require call is broken down to a context and a request. The context is the directory of the module containing the require call and is used to resolve requests with relative paths. For your entry module(s) you need to specify that in the config.

Since compiled webpack bundles don't have any contexts and only use module ids, dynamic requires create a "webpack context" to mock contexts within the bundle based on a mapping.

During compilation, webpack will

  1. resolve the request (which is the static prefix inside the require statement, e.g. "./lang") to a resource (full path) using the context of the require statement
  2. find and include all files that match the filter within that resource path
  3. create a "context module" for those files to replace the dynamic require

Before and after the first step, we can use ContextReplacementPlugin to replace the request/resource, the filter and the subdirectory flag by matching the request or resource. Unfortunately, the IgnorePlugin only runs before the first step and only allows matching the original request, not the context.
To make this simpler, perhaps the IgnorePlugin should allow matching the resolved resource as well, just like the ContextReplacementPlugin, so that this would work:

 new webpack.IgnorePlugin(/moment[\/\\]lang$/)
@sokra

This comment has been minimized.

Copy link
Member

commented Jul 26, 2014

It wasn't in the documentation, but the IgnorePlugin has a second parameter which is a RegExp for the context. (Added it now to the documentation)

So you could do this:

new webpack.IgnorePlugin(/^\.\/lang$/, /moment$/)

which is equal to the custom plugin @taurose proposed.


But we are talking about 160 unminimized bytes. This seem to be nothing compared to the moment.js lib with 85 unminimized _kilo_bytes... 😄

sokra added a commit that referenced this issue Jul 26, 2014

@jdivock

This comment has been minimized.

Copy link

commented Nov 6, 2014

Random ProTip™ for anyone just stumbling into this, with moment 2.0 the folder is called locale and not lang. So the IgnorePlugin line is

 new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/),
@FoxxMD

This comment has been minimized.

Copy link

commented Mar 6, 2015

thanks @jdivock 👍

@jhnns

This comment has been minimized.

Copy link
Member

commented Mar 7, 2015

ProTips™ 👍

@voor

This comment has been minimized.

Copy link

commented Apr 22, 2015

Thanks @jdivock 👍

@Sinewyk

This comment has been minimized.

Copy link

commented May 22, 2015

I'm using the noParse options.

module: {
  noParse: [/moment.js/]
}

according to the doc http://webpack.github.io/docs/configuration.html#module-noparse

The files are expected to have no call to require, define or similar. They are allowed to use exports and module.exports. Which means that the file is not parsed using the regex that includes everything, but it still gets exported. It's pretty much the perfect behavior.

One should use this option for a browser build and require elsewhere the locale data they want, or remove the noParse server side.

@burabure

This comment has been minimized.

Copy link
Contributor

commented Jul 20, 2015

Thanks @Sinewyk ! Saved 150kb. BTW if you need locales or addons require 'moment/locale/..'

@Sinewyk

This comment has been minimized.

Copy link

commented Jul 20, 2015

By the way, for those interested, this behavior (full dynamic require) that was true by default, will become false by default in webpack@2 #729.

In the long run it should be good. At least no more unexpected bundling like moment.js

@fresheneesz

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2015

module: {noParse: [/moment.js/]} seems to bundle moment as-is, which causes it to be added as a global variable. This implies to me that it disables regular static requires too. Not ideal.

@fresheneesz

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2015

Looks like new webpack.ContextReplacementPlugin(/.*$/, /NEVER_MATCH^/) does the trick.

@Daniel15

This comment has been minimized.

Copy link

commented Mar 21, 2016

Thanks @fresheneesz, using a regular expression that will never match is a nice trick to avoid dynamic requires! I'm glad to hear they'll be disabled by default in Webpack 2 (as per #729).

@maciej-gurban

This comment has been minimized.

Copy link

commented Mar 24, 2016

For those who stumble upon this from google like me: whether your issue is locale files not being included, or unwanted locale being included, you can update your bower.json instead of modifying your build.

...
"overrides": {
  "moment": {
      "main": [
        "moment.js",
        "locale/en.js",
        "locale/de.js"
      ]
    }
}
...
@jjalonso

This comment has been minimized.

Copy link

commented Apr 6, 2016

Im a bit confused about all the conversation, I think this is matching my problem but Is hard to understand if the regular expression is to AVOID bundling and do it in runtime or the opposite.

I have to bundle everything in my project BUT NOT the requires that request files under a specific folder.

Of course im working in NODE, then i want to use the default node require. I want webpack to ignore it.

ex:

require('./myfile.js') // Bundle it!
require('./dinamics/myfile.js') // Import it in runtime
@jhnns

This comment has been minimized.

Copy link
Member

commented Apr 8, 2016

@jjalonso With resolve.alias you can rewrite paths. Then you can just rewrite all paths that point to a file inside dinamics to an empty dummy file

@jjalonso

This comment has been minimized.

Copy link

commented Apr 12, 2016

@jhnns Thanks for the answer, I took another pattern but is good to know for a future, thanks you!

@L3V147H4N

This comment has been minimized.

Copy link

commented May 25, 2016

@sokra disabling expression require doesn't seem to work for me

{
  module: {
    // require
    unknownContextRegExp: /$^/,
    unknownContextCritical: false,

    // require(expr)
    exprContextRegExp: /$^/,
    exprContextCritical: false,

    // require("prefix" + expr + "surfix")
    wrappedContextRegExp: /$^/,
    wrappedContextCritical: false
  }
}

I tried this but nothing changed, webpack keeps trying to handle it

my code looks like:

var scriptPath = path.join(process.cwd(), "datafiles", "scripts", "public", req.params[0]);

var Script = require(scriptPath);

this code is executed with every request and requires a particular file, but as webpack is trying to wrap it the module is not found

the code after webpack is

var scriptPath = _path2.default.join(process.cwd(), "datafiles", "scripts", "public", req.params[0]);

var Script = __webpack_require__(34)(scriptPath);

I should point out that I'm using webpack to bundle my server files and then bundling it with Electron.

@handycode

This comment has been minimized.

Copy link

commented Jun 29, 2016

thanks @Sinewyk !

webpack:

module:  {
  ...
  noParse: [/moment.js/]
}

your code

import moment from "moment"
import "moment/locale/fr"
moment.locale("fr")

or 

const moment = require("moment")
require("moment/locale/fr")
moment.locale("fr")

it's nice

@fergardi

This comment has been minimized.

Copy link

commented Nov 16, 2017

How about ignoring all locales with IgnorePlugin except some of my choosing? For example, ignore all except EN, ES, FR. Can that be done with a regular expression?

plugins: [
  new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/)
]

Edit: I figured It out, in case someone still needs it. Found in https://stackoverflow.com/a/25426019/2477303

plugins: [
  new webpack.ContextReplacementPlugin(/moment[\/\\]locale$/, /en|es|fr/),
  // new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/)
]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.