Allow force excluding deep dependencies

[HyperBrain]

This is a Feature Proposal

Description

Currently the forceExclude option in the plugin only excludes 1st level dependencies.
The question often arises if it could also exclude dependencies deeply, e.g.. if you force exclude
the aws-sdk then it would exclude it even if a 3rd dependency references it.

Referencing issues are: #292 and #305

The whole feature is quite tricky, see the discussions in the referenced issues.

The first approach that should be investigated is mentioned in #305 (comment) (multi-pass approach) as it might integrate nicely into the isolation concept of the node_modules folder and lets npm do the work.

[bazaglia]

+1!

[efoncubierta]

Hi,

Has any progress been made on this? I am hitting the same problem.

This is my project structure:

• project/
  • api/
    • src/
    • package.json
    • serverless.yml
    • webpack.config.js
  • common/
    • src/
    • package.json

common is a module which contains common code that will be reused in other services. This common module depends on the aws-sdk module. Obviously, the api module depends on common. When I package the lambda functions in the api module, dev dependencies in common (i.e. aws-sdk) are packaged too.

[46] ../common/node_modules/aws-sdk/lib/aws.js 134 bytes {0} [built]

I tried adding aws-sdk as a peerDependency in common with no luck. I don't know what else to try. It is not affecting functionality, just making the zip files big. Not great.

Thanks.

[craigtsmith]

For anyone else hitting this issue (especially with aws-sdk) I've found that using webpacks 's externals config directly in your webpack.config.js is a pretty clean way of removing aws-sdk from the bundle and ensuring that even dependencies of dependencies that reference aws-sdk maintain their require('aws-sdk') (so that it is loaded from the lambda's execution environment - which was my requirement)

No excludes or includes needed in the serverless.yml:

custom:
  webpack:
    packager: 'yarn'

In the webpack config:

module.exports = {
  entry: slsw.lib.entries,
  mode: 'production',
  externals: [{'aws-sdk': 'commonjs aws-sdk'}]
...

YMMV, might also help help people following: #247 or: #292

[hsz]

I was able to get rid of aws-sdk package which is nested dependency (production dependency for some of my dependencies) with:

custom:
  webpack:
    includeModules: true
    packagerOptions:
      scripts:
        - rm -rf node_modules/aws-sdk

[ffxsam]

When I work with serverless and Lambda, I just add aws-sdk as a dev dependency, and that works fine. I can test locally, and when I deploy, it doesn't package up AWS.

[matthewmorgan]

@ffxsam devDependencies did not resolve the issue for me. I found I needed to explicitly exclude in the webpack config as per @hsz.

[matthewmorgan]

I'm still investigating this and will report back if I find anything helpful in my workflow.

[dawez]

Hey do we have any progress on that? I am also using a package which uses inside aws-sdk so at the end it bumps up the size of the deployment by lot of MB. The solution that @hsz wrote seems that does not change the zip output that will be uploaded to AWS. Trying to add

externals: ["aws-sdk"], or variation of that to the webpack config causes the error:

ERROR Error: Cannot find module 'aws-sdk'

In a function that is uploading to S3, import of the function below:

import * as aws from "aws-sdk";
import { generateRandomAlphanumericString } from "../../lib/utils/uuid";

const s3 = new aws.S3({ apiVersion: "2006-03-01" });

As far as I know the lambda should automatically include the aws-sdk so now I'm lost.

[danrivett]

I was able to get rid of aws-sdk package which is nested dependency (production dependency for some of my dependencies) with:

custom:
  webpack:
    includeModules: true
    packagerOptions:
      scripts:
        - rm -rf node_modules/aws-sdk

Rather than doing the above, the following worked for me until I included the uuid package as a dependency:

custom:
  webpack:
    includeModules:
      forceExclude:
        - aws-sdk

However after including uuid, aws-sdk started being included and so I tried your approach. Unfortunately it didn't work initially, but I found a variation of your approach did.

I described it in detail in this comment to #305.

[somq]

Better delete all aws-sdk folders recursively to avoid skipping nested aws-sdk folders:

    packagerOptions:
      scripts:
        - find . -type d -name 'aws-sdk' -exec rm -rf {} +