"Module not found" warnings when importing AWS s3 in app directory

[rijk]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.5.0: Thu Jun  8 22:21:34 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112
    Binaries:
      Node: 16.18.1
      npm: 8.19.2
      Yarn: N/A
      pnpm: 8.6.3
    Relevant Packages:
      next: 13.4.8-canary.8
      eslint-config-next: 13.4.7
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.1.6
    Next.js Config:
      output: N/A

Which area(s) of Next.js are affected? (leave empty if unsure)

App directory (appDir: true), Package manager (npm, pnpm, Yarn)

Link to the code that reproduces this issue or a replay of the bug

https://github.com/rijk/repro-s3-import-issue

To Reproduce

1. Add an S3 user / key to the .env file (I'm sorry, I tried adding credentials here but AWS didn't let me)
2. Run the app
3. Go to http://localhost:3000/api/test-page
4. Observe signed URL returned without warnings
5. Go to http://localhost:3000/api/test-app
6. Signed URL returned, but with a bunch of Module not found warnings in the console

Describe the Bug

When importing from the s3 SDK in the app directory (either in a route handler or server action), this triggers a bunch of warnings (see below). This doesn't happen in the pages directory. Is module resolution different between the two?

Warnings:

- warn ./node_modules/.pnpm/@aws-sdk+signature-v4-multi-region@3.357.0/node_modules/@aws-sdk/signature-v4-multi-region/dist-cjs/SignatureV4MultiRegion.js
Module not found: Can't resolve '@aws-sdk/signature-v4-crt' in '/Users/Rijk/Code/Playground/next/repro-s3-import-issue/node_modules/.pnpm/@aws-sdk+signature-v4-multi-region@3.357.0/node_modules/@aws-sdk/signature-v4-multi-region/dist-cjs'

Import trace for requested module:
./node_modules/.pnpm/@aws-sdk+signature-v4-multi-region@3.357.0/node_modules/@aws-sdk/signature-v4-multi-region/dist-cjs/SignatureV4MultiRegion.js
./node_modules/.pnpm/@aws-sdk+signature-v4-multi-region@3.357.0/node_modules/@aws-sdk/signature-v4-multi-region/dist-cjs/index.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/runtimeConfig.shared.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/runtimeConfig.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/S3Client.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/index.js
./app/api/test-app/route.ts

./node_modules/.pnpm/@aws-sdk+util-user-agent-node@3.357.0/node_modules/@aws-sdk/util-user-agent-node/dist-cjs/is-crt-available.js
Module not found: Can't resolve 'aws-crt' in '/Users/Rijk/Code/Playground/next/repro-s3-import-issue/node_modules/.pnpm/@aws-sdk+util-user-agent-node@3.357.0/node_modules/@aws-sdk/util-user-agent-node/dist-cjs'

Import trace for requested module:
./node_modules/.pnpm/@aws-sdk+util-user-agent-node@3.357.0/node_modules/@aws-sdk/util-user-agent-node/dist-cjs/is-crt-available.js
./node_modules/.pnpm/@aws-sdk+util-user-agent-node@3.357.0/node_modules/@aws-sdk/util-user-agent-node/dist-cjs/index.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/runtimeConfig.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/S3Client.js
./node_modules/.pnpm/@aws-sdk+client-s3@3.362.0/node_modules/@aws-sdk/client-s3/dist-cjs/index.js
./app/api/test-app/route.ts

Expected Behavior

The code should run without warnings, as it does in the pages directory.

Which browser are you using? (if relevant)

not relevant

How are you deploying your application? (if relevant)

not relevant

[balazsorban44]

Hi, this is similar to #52091 (comment)

The current solution is:

/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: ['@aws-sdk/client-s3'],
  },
}

module.exports = nextConfig

[rijk]

That was it, thank you 🙏

For me specifically, I had to add two packages:

/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: [
      '@aws-sdk/client-s3',
      '@aws-sdk/s3-presigned-post',
    ],
  },
}

module.exports = nextConfig

Just a note, and not meant as criticism but it might save you work in the future: I've read that documentation page in the past, but I honestly never connected it to this type of issue. Maybe it would be good to expand this page with:

1. An explanation why "Dependencies will automatically be bundled by Next.js", how this works technically and what the benefit is;
2. What issues you might encounter which indicate that you'd need to exclude them (maybe some example error messages, so people Googling those will find this page).

Especially because for popular packages like Prisma this is done automatically, it can be extra confusing when it's not working.

[balazsorban44]

Thanks! We are discussing this internally to improve both the messaging, and hopefully also the automatic detection.

[github-actions]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.