Strange bundling error when including sub directory

[DavidWells]

Including a file from a sub folder is crashing my netlify deploy with this error

. Hashing files...Error: Could not find "encoding" module in file: /unsubscribe-center/functions/redirect.js.
Please ensure "encoding" is installed in the project.
    at handle (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/finders.js:76:17)
    at precinct.paperwork.forEach.dependency (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/finders.js:102:11)
    at Array.forEach (<anonymous>)
    at getDependencies (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/finders.js:95:8)
    at filesForFunctionZip (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/zip.js:65:7)
    at zipJs (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/zip.js:110:3)
    at zipFunction (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/zip.js:143:12)
    at ~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/@netlify/zip-it-and-ship-it/src/zip.js:181:9
    at element (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/p-all/index.js:4:67)
    at Promise.resolve.then.element (~/.nvm/versions/node/v10.4.1/lib/node_modules/netlify-cli/node_modules/p-map/index.js:47:21)
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! unsubscribe-center@0.1.0 deploy: `npm run build && netlify deploy -p`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the unsubscribe-center@0.1.0 deploy script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/davidwells/.npm/_logs/2019-03-08T00_15_27_349Z-debug.log

The file is in a sub folder and has all the dependancies installed locally.

Here is the dep tree

I'm using node-fetch as a dependancy in the /functions/customerio/api/index.js file

[biilmann]

hmm, where's the encoding module imported?

[DavidWells]

No idea. I'm just importing node-fetch. Somewhere deep down the tree perhaps.

├── node-fetch@2.3.0
└─┬ simple-oauth2@2.2.1
  ├── date-fns@1.30.1
  ├─┬ debug@3.2.6
  │ └── ms@2.1.1
  ├─┬ joi@13.7.0
  │ ├── hoek@5.0.4
  │ ├─┬ isemail@3.2.0
  │ │ └── punycode@2.1.1
  │ └─┬ topo@3.0.3
  │   └── hoek@6.1.2
  └─┬ wreck@14.2.0
    ├─┬ boom@7.3.0
    │ └── hoek@6.1.2
    ├── bourne@1.1.2
    └── hoek@6.1.2

The modules aren't corrupted, the functions deploy fine if they don't require files from sibling folders.

I ended up hoisting everything to the parent folder for now to get around this

[bcomnes]

Did this get fixed?

[DavidWells]

Not yet. I fixed this by reorganizing my file structure.

Seemed like module resolution was not looking in sibling folder for dep. Which I think is the correct behavior.

Haven't see this issue since I reorganized my files

[zanona]

Also having this issue now, but it started from nothing, no change to netlify functions codebase whatsoever. In fact deploy previews were passing without issues and when it got merged to production, this poped up.

[bcomnes]

The solution is to install encoding. Its an undeclared peer dependency of node-fetch. Our tree shaker may be a little to aggressive right now, but thats the solution for now.

[Auspicus]

Bump. Also ran into this issue with cross-fetch which uses node-fetch.

[ehmicky]

This should be fixed by d3e4b2a, please let us know if this is still happening.

[polarbirke]

Just happened to me on a preview deployment with node-fetch being a dependency of my lambda function.

[jon301]

Hi

I'm encountering a similar error (not encoding but @nestjs/microservices module) when trying to deploy a Function exposing a NestJS app bundled with Nx :

11:43:52 PM: Function Dir: /opt/build/repo/dist/apps/api-lambda
11:43:52 PM: TempDir: /tmp/zisi-5db619af1f4f9f00071d3713
11:43:53 PM: Prepping functions with zip-it-and-ship-it 0.4.0-5
11:43:56 PM: Error: In file "/opt/build/repo/dist/apps/api-lambda/main.js": Cannot find module '@nestjs/microservices' from '/opt/build/repo/node_modules/@nestjs/core'
11:43:56 PM: Error prepping functions

More info in this comment from @kamilmysliwiec (creator of NestJS) in this thread : nestjs/nest#1706 (comment)

TLDR :

webpack traverse modules graph and tries to resolve cache-manager dependency even though(!) CacheModule might not be used (tree-shaking is being performed later on, once bundle is created). Consequently, it breaks Nest idea of lazy require() calls, because webpack tries to load them all during the bundling process

From what I saw later in the thread, there are potentially other lazy imports :

      const lazyImports = [
        '@nestjs/microservices',
        '@nestjs/microservices/microservices-module',
        '@nestjs/websockets',
        '@nestjs/websockets/socket-module',
        '@nestjs/platform-express',
        'cache-manager',
        'class-validator',
        'class-transformer',
      ];

Is there a way to tell zip-it-and-ship-it to ignore those dependencies ?

Installing all those peerDependencies as suggested by @bcomnes #16 (comment) could work, but there are plenty + not sure this is an exhaustive list (could break anytime if NestJS code evolves)

[ehmicky]

Hi @jon301 and @polarbirke. I've just made a new release. Could you please check if it now works?

Also you should see the following line in your log:

Prepping functions with zip-it-and-ship-it 0.4.0-6

If not, please let me know. Thanks!

[jon301]

Hi @ehmicky, thanks for checking this out
I've just tried to trigger new deployment but it's still running in the 0.4.0-5 version :

6:36:59 PM: Prepping functions with zip-it-and-ship-it 0.4.0-5
6:37:02 PM: Error: In file "/opt/build/repo/dist/apps/api-lambda/main.js": Cannot find module '@nestjs/microservices' from '/opt/build/repo/node_modules/@nestjs/core'
6:37:02 PM: Error prepping functions

[ehmicky]

Can you try again? You should now see:

Prepping functions with zip-it-and-ship-it 0.3.1

And your initial issue should now be solved.

[jon301]

Tried just now and still with old version :(

11:54:28 PM: Prepping functions with zip-it-and-ship-it 0.4.0-5

[jon301]

Hi @ehmicky , thanks for your help
It seems like the previous error is gone with 0.4.0-6, but a new one appeared :

8:02:59 AM: Prepping functions with zip-it-and-ship-it 0.4.0-6
8:03:02 AM: Error: In file "/opt/build/repo/dist/apps/api/main.js": Cannot find module 'graphql-tag-pluck/package.json' from '/opt/build/repo/node_modules/graphql-toolkit'

Edit : I think it's because of this line : https://github.com/ardatan/graphql-toolkit/blob/master/packages/loaders/code-file/src/extract-document-string-from-code-file.ts#L52

[ehmicky]

Since that's a separate issue, I have opened a new issue at #66.

It looks like the original issue is solved. Please let me know if this appears again.

[ehmicky]

This issue is symptomatic of a more general problem described at #68. Please refer to this issue for further discussion.

[EdwardGoomba]

The solution is to install encoding. Its an undeclared peer dependency of node-fetch. Our tree shaker may be a little to aggressive right now, but thats the solution for now.

Thanks - ran into this issue. Resolved.