Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@aws-sdk/client-s3 makes node_modules too large #2747

Closed
remagpie opened this issue Sep 3, 2021 · 6 comments
Closed

@aws-sdk/client-s3 makes node_modules too large #2747

remagpie opened this issue Sep 3, 2021 · 6 comments
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@remagpie
Copy link

remagpie commented Sep 3, 2021

Describe the bug

When @aws-sdk/client-s3@3.29.0 is installed, it downloads aws-crt as a dependency which is about 200MB in size. It occupies about 60MB even when we compress it.

My team updates the lambda function by directly zipping the directory and uploading it, but we can't do it anymore because we hit the size limit of the allowed archive size.

The problem is gone after we downgrade @aws-sdk/client-s3 to 3.28.0, but we'd like to use the most recent version if possible.

Your environment

SDK version number

@aws-sdk/client-s3@3.29.0

Is the issue in the browser/Node.js/ReactNative?

Node.js

Additional context

@remagpie remagpie added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 3, 2021
@moltar
Copy link

moltar commented Sep 3, 2021

After bundling using esbuild and deploying to Lambda we are getting the following errors:

AWS CRT binary not present in any of the following locations:\n\t/bin/native/aws-crt-nodejs\n\t/bin/linux-x64/aws-crt-nodejs
    "stack": [
        "Error: AWS CRT binary not present in any of the following locations:",
        "\t/bin/native/aws-crt-nodejs",
        "\t/bin/linux-x64/aws-crt-nodejs",
        "    at null.node_modules/aws-crt/dist/native/binding.js (/node_modules/aws-crt/lib/native/binding.js:46:11)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/aws-crt/dist/native/crt.js (/node_modules/aws-crt/lib/native/crt.ts:18:1)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/aws-crt/dist/index.js (/node_modules/aws-crt/lib/index.ts:17:1)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/@aws-sdk/signature-v4-crt/dist/cjs/CrtSignerV4.js (/node_modules/@aws-sdk/signature-v4-crt/src/CrtSignerV4.ts:22:1)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/@aws-sdk/signature-v4-crt/dist/cjs/index.js (/node_modules/@aws-sdk/signature-v4-crt/src/index.ts:1:1)",
        "    at __require (/var/task/index.js:41:45)"
    ]

@ml1nk
Copy link

ml1nk commented Sep 3, 2021

In addition since aws-crt includes a native blob which does not support alpine, it fails for example on node:16.8.0-alpine.

ERROR: Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/main/node_modules/aws-crt/dist/bin/linux-x64/aws-crt-nodejs.node)

@webberwang
Copy link

webberwang commented Sep 3, 2021

I notice Cannot find module '/node_modules/aws-crt/dist/bin/darwin-x64/aws-crt-nodejs' from '/node_modules/aws-crt/dist/native/binding.js' in 3.29.0 (which isn't fixed even if I install the missing package), had to downgrade to 3.28.0

@matheuscanela
Copy link

matheuscanela commented Sep 6, 2021

The s3 adds 209 packages as it gets removed after removing the client-s3
Cannot be used with lambda, as it consumes 200+ mb

@AllanZhengYP
Copy link
Contributor

Thank you for reporting this issue. We are investigating in the issue and will post update in #2750

@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

6 participants