Webpack error when adding plugin

[gusfune]

I'm installing this plugin on a fresh install of Payload CMS, but I get the following error when adding to the configuration

assets by status 3.9 MiB [cached] 22 assets
Entrypoint main = styles.css styles.736a74d45b8bb6de2c0e.js main.ae9d49edb45563f3f421.js 18 auxiliary assets
orphan modules 2.54 MiB [orphan] 1295 modules
runtime modules 31.4 KiB 15 modules
cacheable modules 6.48 MiB (javascript) 190 KiB (css/mini-extract) 4.37 KiB (asset)
  javascript modules 6.19 MiB 1024 modules
  css modules 190 KiB
    modules by path ./node_modules/payload/dist/admin/components/ 139 KiB 162 modules
    + 3 modules
  json modules 301 KiB
    ./node_modules/payload/dist/translations/cs.json 14 KiB [built] [code generated]
    ./node_modules/payload/dist/translations/de.json 15.7 KiB [built] [code generated]
    ./node_modules/payload/dist/translations/en.json 14.1 KiB [built] [code generated]
    + 18 modules
  asset modules 84 bytes (javascript) 4.37 KiB (asset)
    ./node_modules/payload/dist/admin/assets/images/favicon.svg 42 bytes (javascript) 437 bytes (asset) [built] [code generated]
    ./node_modules/payload/dist/admin/assets/images/og-image.png 42 bytes (javascript) 3.94 KiB (asset) [built] [code generated]

ERROR in ./node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/handleUpload.js 43:27-40
Module not found: Error: Can't resolve 'fs' in '/Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3'
resolve 'fs' in '/Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3'
  Parsed request is a module
  using description file: /Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/package.json (relative path: ./dist/adapters/s3)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      /Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/node_modules doesn't exist or is not a directory
      /Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/node_modules doesn't exist or is not a directory
      /Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/dist/node_modules doesn't exist or is not a directory
      /Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/plugin-cloud-storage/node_modules doesn't exist or is not a directory
      /Users/miiuser/Developer/services/apps/payloadcms/node_modules/@payloadcms/node_modules doesn't exist or is not a directory
      /Users/miiuser/Developer/services/apps/payloadcms/node_modules/node_modules doesn't exist or is not a directory
      looking for modules in /Users/miiuser/Developer/services/apps/payloadcms/node_modules
        single file module
          using description file: /Users/miiuser/Developer/services/apps/payloadcms/package.json (relative path: ./node_modules/fs)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/fs doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/fs.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/fs.tsx doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/fs.js doesn't exist
            .json
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/fs.json doesn't exist
        /Users/miiuser/Developer/services/apps/payloadcms/node_modules/fs doesn't exist
      /Users/miiuser/Developer/services/apps/node_modules doesn't exist or is not a directory
      looking for modules in /Users/miiuser/Developer/services/node_modules
        single file module
          using description file: /Users/miiuser/Developer/services/package.json (relative path: ./node_modules/fs)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/node_modules/fs doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/node_modules/fs.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/node_modules/fs.tsx doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/node_modules/fs.js doesn't exist
            .json
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/node_modules/fs.json doesn't exist
        /Users/miiuser/Developer/services/node_modules/fs doesn't exist
      /Users/miiuser/Developer/node_modules doesn't exist or is not a directory
      looking for modules in /Users/miiuser/node_modules
        single file module
          No description file found in /Users/miiuser/node_modules or above
          no extension
            Field 'browser' doesn't contain a valid alias configuration
            /Users/miiuser/node_modules/fs doesn't exist
          .ts
            Field 'browser' doesn't contain a valid alias configuration
            /Users/miiuser/node_modules/fs.ts doesn't exist
          .tsx
            Field 'browser' doesn't contain a valid alias configuration
            /Users/miiuser/node_modules/fs.tsx doesn't exist
          .js
            Field 'browser' doesn't contain a valid alias configuration
            /Users/miiuser/node_modules/fs.js doesn't exist
          .json
            Field 'browser' doesn't contain a valid alias configuration
            /Users/miiuser/node_modules/fs.json doesn't exist
        /Users/miiuser/node_modules/fs doesn't exist
      /Users/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory
      looking for modules in /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules
        single file module
          using description file: /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/package.json (relative path: ./node_modules/fs)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules/fs doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules/fs.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules/fs.tsx doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules/fs.js doesn't exist
            .json
              Field 'browser' doesn't contain a valid alias configuration
              /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules/fs.json doesn't exist
        /Users/miiuser/Developer/services/apps/payloadcms/node_modules/payload/node_modules/fs doesn't exist
 @ ./node_modules/@payloadcms/plugin-cloud-storage/dist/adapters/s3/index.js 31:21-46
 @ ./node_modules/@payloadcms/plugin-cloud-storage/s3.js 1:0-59
 @ ./src/payload.config.ts 5:0-64 6:14-23
 @ ./node_modules/payload/dist/admin/Root.js
 @ ./node_modules/payload/dist/admin/index.js 10:31-48

webpack 5.80.0 compiled with 1 error in 7326 ms
❯ payloadcms (DV-1647-create-a-database-schema-in-payload-cms-for-our-products) ✘ 

I tried to polyfill fs with path-browserify, but no success. All the configurations are default.
Interestingly, it seems to be looking for node_modules inside a node_modules.
Running it on yarn workspace, but, tried to isolate to a non-workspace and same issue happens.
Node 18.19.0

[jmikrut]

Hey @gusfune — so here's a quick description of what's happening.

The Payload config itself combines server-only code and browser-only code into one file, which lets us keep everything simple and allows the config to be the single source of truth for how Payload gets built up.

But, with that comes the challenges of making sure that server-side code gets excluded within the browser JS bundle. As you mentioned, polyfilling is a potential way to do this. But, this plugin actually takes care of this for you as long as it is initialized properly.

If you look at the adapters for each of our supported cloud storage vendors, each one "extends" Payload's webpack config for you so that you don't have to do anything, and the server-side code gets excluded from your bundle automatically.

I think this may be a case of not initializing the plugin appropriately, thus it is not successfully adding its aliases to the Webpack config.

Can you send me your Payload config code? Are you passing the plugin accurate arguments?

[gusfune]

Thanks for the quick response @jmikrut - seems like the issue was trying to make the configuration "customisable" by making the plugin option, kinda like:

const plugins = []

const S3_ACCESS_KEY_ID = process.env.S3_ACCESS_KEY_ID
const S3_SECRET_ACCESS_KEY = process.env.S3_SECRET_ACCESS_KEY
const S3_BUCKET = process.env.S3_BUCKET

if (
  process.env.NODE_ENV !== "development" &&
  S3_ACCESS_KEY_ID &&
  S3_SECRET_ACCESS_KEY &&
  S3_BUCKET
) {
  plugins.push(
    cloudStorage({
      collections: {
        media: {
          adapter: s3Adapter({
            config: {
              credentials: {
                accessKeyId: process.env.S3_ACCESS_KEY_ID,
                secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
              },
            },
            bucket: process.env.S3_BUCKET,
          }),
          disableLocalStorage: true,
        },
      },
    })
  )
}

export default buildConfig({
  serverURL: process.env.SERVER_URL,
  admin: {
    user: Users.slug,
  },
  collections: [Users, Media, Products],
  localization: {
    locales: ["ar", "en", "de", "nl", "zh"],
    defaultLocale: "en",
    fallback: true,
  },
  typescript: {
    outputFile: path.resolve(__dirname, "payload-types.ts"),
  },
  upload: {
    limits: {
      fileSize: 5000000, // 5MB, written in bytes
    },
  },
  graphQL: {
    schemaOutputFile: path.resolve(__dirname, "generated-schema.graphql"),
  },
  plugins,
})

By just adding everything to the config, it compiles. I am closing this, but leave the suggestion to maybe allow disabling the remote storage on local development mode.

[omavi]

I am closing this, but leave the suggestion to maybe allow disabling the remote storage on local development mode

Agreed, @jmikrut is there any guidance on local testing without needing to modify the code? Right now I have to completely remove the s3Adapter declaration, even if it's in a short circuited statement like the following:

import { s3Adapter } from '@payloadcms/plugin-cloud-storage/s3';

const storageAdapter =
  process.env.NODE_ENV === 'development'
    ? null
    : s3Adapter({
        config: {
          region: String(process.env.AWS_REGION),
          credentials: {
            accessKeyId: String(process.env.AWS_ACCESS_KEY_ID),
            secretAccessKey: String(process.env.AWS_SECRET_ACCESS_KEY),
          },
        },
        bucket: String(process.env.AWS_S3_BUCKET),
      });

export { storageAdapter };

I also tried using the localstack s3 setup but it looks like persistence is a paid feature now.

[omavi]

I worked around this by always passing the s3Adapter even though none of the env vars have values in dev. Thankfully this caused no errors I could see. Then I updated my config like so:

const disableLocalStorage = process.env.NODE_ENV !== 'development';
const generateFileURL: GenerateFileURL = ({ collection, filename }) => {
  return (
    ensureEndsWith(
      disableLocalStorage
        ? String(process.env.CDN_BASE_URL)
        : `/${collection.slug}`,
      '/',
    ) + filename
  );
};

const disablePayloadAccessControl = true;

export default buildConfig({
  ...
  plugins: [
    ...
    cloudStorage({
      collections: {
        media: {
          adapter: storageAdapter,
          generateFileURL,
          disableLocalStorage,
          disablePayloadAccessControl,
        },
        files: {
          adapter: storageAdapter,
          generateFileURL,
          disableLocalStorage,
          disablePayloadAccessControl,
        },
      },
    }),
  ],
});

Not ideal, but it works ¯\_(ツ)_/¯