New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some modules are missing when using webpack 2 #116

Closed
HyperBrain opened this Issue Mar 27, 2017 · 8 comments

Comments

Projects
None yet
4 participants
@HyperBrain
Copy link
Member

HyperBrain commented Mar 27, 2017

Problem

We tried to use Webpack 2 and the current master branch of the webpack plugin.
The bundle generation is creating a correct bundle, but the packaged node modules,
that are installed as sibling to the bundle by the plugin are missing some external modules.

I.e. although they are present in the local services' package.json, they do not appear in
the package.json that is temporarily created by the webpack plugin to install the needed
external modules.

This leads to "required module not found" errors, when executing the compiled code on AWS
after Serverless deployed it.

Root cause

The plugin parses the webpack stats to find external modules by matching the returned stats
with "external ".

See comment below: Webpack 2's dependency optimization sometimes removes first level dependencies but keeps the removed dep's second level dependencies.

@HyperBrain

This comment has been minimized.

Copy link
Member Author

HyperBrain commented Mar 27, 2017

I'll have to investigate this further. The package.json created by the plugin in .webpack is different regarding the dependencies than the original package.json. I'll reopen this as soon as I have more qualified information at hand.
Seems to be more than just the npm install.

@HyperBrain HyperBrain closed this Mar 27, 2017

@HyperBrain HyperBrain changed the title Node 6 does not include second level dependencies Modules are missing when using webpack 2 Mar 27, 2017

@HyperBrain HyperBrain reopened this Mar 27, 2017

@HyperBrain

This comment has been minimized.

Copy link
Member Author

HyperBrain commented Mar 27, 2017

Reopened the issue after deeper investigation. Changed the description and subject to match my findings and the problem.

@HyperBrain HyperBrain changed the title Modules are missing when using webpack 2 Some modules are missing when using webpack 2 Mar 27, 2017

@HyperBrain

This comment has been minimized.

Copy link
Member Author

HyperBrain commented Mar 27, 2017

Further analysis:

The stats object returned by webpack 2 contains all external references marked with "external ", including the second level dependencies.
It happens that webpack optimizes the imports so, that a first level dependency does not get imported, but one of its second level dependencies does.

This happened for us with react-addons-css-transition-group.
The primary dependency got optimized away, but instead it only references react-addons-transition-group which is a second level dependency of the css transition group module.

The plugin grabs all exported dependencies but cannot npm install the react-addons-transition-group because only its parent is mentioned in our package.json. So the (needed) dependency is missing in the resulting node modules.

A solution would be, that the module lookup is improved, so that each dependency reported by webpack 2 will be looked up with a deep search, beginning from the main service package.json throughout all of the (production) dependencies' dependencies. This would make sure that webpack 2's optimization will work with the plugin.

@HyperBrain

This comment has been minimized.

Copy link
Member Author

HyperBrain commented Mar 28, 2017

It is even more complex. The new webpack 2 optimizer now emits partial modules as external references:

external "aws-sdk/global"
external "codemirror/addon/lint/javascript-lint"
external "codemirror/addon/lint/lint"
external "codemirror/addon/lint/yaml-lint"
external "codemirror/mode/javascript/javascript"
external "codemirror/mode/yaml/yaml"
external "crypto"
external "deep-assign"
external "hammerjs"
external "intl"
external "intl/locale-data/jsonp/en-US.js"
external "intl/locale-data/jsonp/pt-BR.js"
external "js-cookie"
external "js-yaml"
external "jshint"
external "markdown-to-jsx"
external "querystring"
external "react-codemirror"
external "react-desc"
external "react-dom/server"
external "react-dropzone"
external "react-facebook-login"
external "react-google-login"
external "react-html-document"
external "redux-thunk"
external "uuid/v4"

This means that the npm install approach will not work anymore, because the referenced externals might be only submodules or files out of an installed module.
The only way to handle this correctly is, in my opinion, copying all of the emitted externals right from the project's node_modules folder as webpack operated on exactly this.

@adambiggs

This comment has been minimized.

Copy link

adambiggs commented Apr 24, 2017

Fixing #71 would also fix this it seems.

@Alex-Mann

This comment has been minimized.

Copy link

Alex-Mann commented May 6, 2017

I am also experiencing a similar issue where I use yarn to install a local npm module (a library I have created) and webpack reports a "Webpack Missing Module 'Module Not Found'". Not sure if this is an error with webpack or this plugin as I didn't have time to dig too deeply

@daviskoh

This comment has been minimized.

Copy link

daviskoh commented Aug 16, 2017

any updates on this?

@HyperBrain

This comment has been minimized.

Copy link
Member Author

HyperBrain commented Aug 16, 2017

@daviskoh I stabilized the packaging in V3 and we run that already in our production environment. You can try it by using version ^3.0.0-rc.1. We did not experience "module losses" anymore.

However, the key to make it work well, is to use the new slsw.lib.entries automatic entry/handler detection and making sure that the function definitions in your serverless.yml point correctly to the handler files if they are located in subdirectories.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment