Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graceful-fs crashes with "RangeError: Maximum call stack size exceeded" #6667

Closed
ElinksFr opened this issue Sep 11, 2019 · 37 comments · Fixed by #6717
Closed

graceful-fs crashes with "RangeError: Maximum call stack size exceeded" #6667

ElinksFr opened this issue Sep 11, 2019 · 37 comments · Fixed by #6717
Assignees

Comments

@ElinksFr
Copy link

ElinksFr commented Sep 11, 2019

❗️ NOTE FROM MAINTAINERS ❗️

This issue is result of relying on non-latest (buggy) graceful-fs version, either directly or through other (not serverless) dependencies. serverless on its own expects latest version of graceful-fs to be installed.

Crash happens when two versions of graceful-fs are initialized and one of them is non-latest (buggy) version.

SOLUTION

Ensure project and its dependencies work only with latest graceful-fs version.

It usually can be solved by nuking node_modules and package-lock.json (or yarn.lock) and reinstalling the project.

When relying on Yarn package manager it can also be solved by adding following instruction to package.json

  "resolutions": {
    "graceful-fs": "4.2.2"
  }

and running yarn install (thanks @sheerun for this tip)


This is a Bug Report

Description

  • What went wrong?
    RangeError: Maximum call stack size exceeded while deploying
  • What did you expect should have happened?
    Deploy without problem
  • What stacktrace or error message from your provider did you see?
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:199:24)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)
      at WriteStream (/opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/graceful-fs.js:201:29)

Similar or dependent issues:

Additional Data

  • Serverless Framework Version you're using: 1.52.0
@medikoo medikoo added the bug label Sep 11, 2019
@medikoo medikoo self-assigned this Sep 11, 2019
@medikoo
Copy link
Contributor

medikoo commented Sep 11, 2019

@ElinksFr does it happen when you're relying on global installation, or when you're invoking local one?

If it's the local one, can you share content of dependencies sections in package.json ?

@medikoo
Copy link
Contributor

medikoo commented Sep 11, 2019

Cause for that is that most likely at /opt/atlassian/pipelines/agent/build/node_modules/graceful-fs/ there's a non-latest graceful-fs installed which shares a bug, that results with such error when multiple instances of graceful-fs are loaded.

Related issue in graceful-fs -> isaacs/node-graceful-fs#174

@medikoo medikoo assigned ElinksFr and unassigned medikoo Sep 11, 2019
@Manzalber
Copy link

Manzalber commented Sep 11, 2019

From our side, we faced the same issue 30 min ago. If i can add some info to it, our packages such as serverless-bundle, etc, are using an outdated version of graceful-fs.
Adding a direct dependency to the same Graceful-fs dependency version Serverless has:
"graceful-fs": "^4.2.2",
It should work for the time being, i know there may be prettier ways to fix it.

@hrnn
Copy link

hrnn commented Sep 11, 2019

It is happening to me as well full logs

Solved by updating package-lock.json as other mentioned here.

@medikoo
Copy link
Contributor

medikoo commented Sep 11, 2019

It is happening to me as well full logs

@hrnn Presented log is unrelated to this issue

@vitorfs
Copy link

vitorfs commented Sep 11, 2019

I started facing the same issue inside a CodeBuild a couple of hours ago. If I roll back to version 1.51.0 it works normally.

[Container] 2019/09/11 15:09:31 Phase complete: PRE_BUILD State: SUCCEEDED 
[Container] 2019/09/11 15:09:31 Phase context status code:  Message:  
[Container] 2019/09/11 15:09:32 Entering phase BUILD 
[Container] 2019/09/11 15:09:32 Running command serverless package --stage $STAGE --verbose 
Serverless: Using Python specified in "runtime": python3.7 
Serverless: Packaging Python WSGI handler... 
Serverless: Generated requirements from /codebuild/output/src942665664/src/eskolare-backend/requirements.txt in /codebuild/output/src942665664/src/eskolare-backend/.serverless/requirements.txt... 
Serverless: Installing requirements from /root/.cache/serverless-python-requirements/f9922aa4f4dfc5b04a7923931a712149_slspyc/requirements.txt ... 
Serverless: Using download cache directory /root/.cache/serverless-python-requirements/downloadCacheslspyc 
Serverless: Running ... 
Serverless: Packaging service... 
Serverless: Excluding development dependencies... 
  
  Range Error -------------------------------------------- 
  
  RangeError: Maximum call stack size exceeded 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:216:24) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 

... +7000 lines 

      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at new WriteStream (/codebuild/output/src942665664/src/node_modules/graceful-fs/graceful-fs.js:218:29) 
      at Object.createWriteStream (/usr/local/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:306:12) 
      at Package.zipFiles (/usr/local/lib/node_modules/serverless/lib/plugins/package/lib/zipService.js:80:23) 
      at resolveFilePathsAll.then.filePaths (/usr/local/lib/node_modules/serverless/lib/plugins/package/lib/packageService.js:120:12) 
      at tryCatcher (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23) 
      at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:517:31) 
      at Promise._settlePromise (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:574:18) 
      at Promise._settlePromise0 (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:619:10) 
      at Promise._settlePromises (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/promise.js:699:18) 
      at _drainQueueStep (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:138:12) 
      at _drainQueue (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:131:9) 
      at Async._drainQueues (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:147:5) 
      at Immediate.Async.drainQueues [as _onImmediate] (/usr/local/lib/node_modules/serverless/node_modules/bluebird/js/release/async.js:17:14) 
      at runCallback (timers.js:705:18) 
      at tryOnImmediate (timers.js:676:5) 
      at processImmediate (timers.js:658:5) 
      at process.topLevelDomainCallback (domain.js:126:23) 
  
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable. 
  
  Get Support -------------------------------------------- 
     Docs:          docs.serverless.com 
     Bugs:          github.com/serverless/serverless/issues 
     Issues:        forum.serverless.com 
  
  Your Environment Information --------------------------- 
     Operating System:          linux 
     Node Version:              10.16.0 
     Framework Version:         1.52.0 
     Plugin Version:            2.0.0 
     SDK Version:               2.1.1 
  
 
[Container] 2019/09/11 15:10:04 Command did not exit successfully serverless package --stage $STAGE --verbose exit status 1 
[Container] 2019/09/11 15:10:04 Phase complete: BUILD State: FAILED 
[Container] 2019/09/11 15:10:04 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: serverless package --stage $STAGE --verbose. Reason: exit status 1 

@medikoo
Copy link
Contributor

medikoo commented Sep 11, 2019

@vitorfs it's most likely caused by fact that at /codebuild/output/src942665664/src/node_modules/graceful-fs there's non latest (and buggy) graceful-fs version installed.

Ensure to update dependencies of the project and issue should be fixed

@calebswank11
Copy link

@vitorfs it's most likely caused by fact that at /codebuild/output/src942665664/src/node_modules/graceful-fs there's non latest (and buggy) graceful-fs version installed.

Ensure to update dependencies of the project and issue should be fixed

~ Hat tip to ya good sir on pointing us in the right direction.

We ran into this issue on a pipeline build and got us head scratching for a bit. If doing this via ADO or Gitlab, ensure that you change the global serverless install during the build.

install -g serverless@1.51.0

@JacksonToomey
Copy link

My org also started getting this error today

@dashmug
Copy link
Contributor

dashmug commented Sep 11, 2019

Same problem here. graceful-fs not being graceful.

@ilima-nvps
Copy link

Also seeing this problem. Rolling back to 1.51.0 solved it for me.

@vitorfs
Copy link

vitorfs commented Sep 11, 2019

At first I did roll back to version 1.51.0 and it fixed the problem for me.

Then I followed @medikoo tip, about graceful-fs being outdated, and solved it by doing something else. I didn't have graceful-fs as a primary dependency, but several other packages depended on it, using different versions.

I updated some packages and re-generated my package-lock.json and it solved my issue. Now I'm successfully using 1.52.0.

mooyoul added a commit to balmbees/node-standard that referenced this issue Sep 11, 2019
@wuelcas
Copy link

wuelcas commented Sep 12, 2019

I had the exact same issue. You'll need to roll back to a previous serverless version or you'll need to make sure graceful-fs is updated to its latest version by updating packages that depend on it (my case were fs-extra and nyc besides serverless). I went for the latter.

@cyberfox1
Copy link

Nothing like testing in production. Jeez.

@medikoo
Copy link
Contributor

medikoo commented Sep 12, 2019

I've updated main description with specific info on how this issue can be solved within a project

@medikoo medikoo changed the title version 1.52.0 RangeError: Maximum call stack size exceeded graceful-fs crashes with "RangeError: Maximum call stack size exceeded" Sep 12, 2019
romainquellec added a commit to CuistotduCoin/front that referenced this issue Sep 12, 2019
@iamfeek
Copy link

iamfeek commented Sep 16, 2019

@medikoo cool. For my case, I cant really do the fix coz I dont even have a project ready.

I believe downgrading is the fastest way to solve my problems.

@radove
Copy link

radove commented Sep 16, 2019

We ran into this issue this morning and added the following to our package.json to resolve the issue for now (npm based):

"dependencies": {
       "graceful-fs": "^4.2.2"
 }

@dre-understand
Copy link

From our side, we faced the same issue 30 min ago. If i can add some info to it, our packages such as serverless-bundle, etc, are using an outdated version of graceful-fs.
Adding a direct dependency to the same Graceful-fs dependency version Serverless has:
"graceful-fs": "^4.2.2",
It should work for the time being, i know there may be prettier ways to fix it.

@Manzalber thank-you. Rolling back to serverless 1.51.0 wasn't an option as I needed fixes in 1.52.0.

@iamchathu
Copy link

I have global installation of latest serverless module and when ever I tried to create a new project with sls create
whatever the template is i keeping getting

  at ReadStream (/Users/user/.config/yarn/global/node_modules/serverless/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:178:28)
      at ReadStream (/Users/user/.config/yarn/global/node_modules/serverless/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:178:28)
      ...

@medikoo
Copy link
Contributor

medikoo commented Sep 18, 2019

@iamchathu from stack trace, it's clear that for some reason Yarn provides an older version of graceful-fs to fs-extra, instead of providing same version that serverless uses.

I believe that what would fix this, is clearing ~/.config/yarn and installing serverless globally again.

@mark-edf
Copy link

the frustrating thing is that serverless deploy doesn't seem to respect the package-lock.json even after I have run npm ci which does.

That meant that our build broke, even though we hadn't done any updates to the package versions.

@medikoo
Copy link
Contributor

medikoo commented Sep 19, 2019

the frustrating thing is that serverless deploy doesn't seem to respect the package-lock.json even after I have run npm ci which does.

Serverless does not install and is not responsible for installing dependencies of your project. It's npm, yarn or whatever package manager you bet on, and it's where package-lock.json (or alterantives) are respected and processed.

@dobesv
Copy link

dobesv commented Sep 19, 2019

Is there a workaround for installing serverless globally? Currently I just ran yarn global install serverless@1.51.0 to downgrade my global version but it would be nice to have a workaround posted for the latest version as well.

@medikoo
Copy link
Contributor

medikoo commented Sep 20, 2019

@dobesv I believe the workaround would be to nuke folder that contains all global installs, and install serverless globally again.

On our side what we can improve to upgrade to latest version of fs-extra, that should also help to minimize the issue. We will look into that.

medikoo added a commit that referenced this issue Sep 20, 2019
@pmuens pmuens added bug and removed question labels Sep 20, 2019
medikoo added a commit that referenced this issue Sep 20, 2019
@pmuens
Copy link
Contributor

pmuens commented Sep 20, 2019

Just a quick update that the v1.52.2 release will provide a fix for this issue --> #6718

medikoo added a commit that referenced this issue Jan 10, 2020
We were fixed on v4.2.1 due to critical bug in v4.2.2
(more info at  #6667) but that's fixed with v4.2.3
medikoo added a commit that referenced this issue Jan 10, 2020
We were fixed on v4.2.1 due to critical bug in v4.2.2
(more info at  #6667) but that's fixed with v4.2.3
richarddd pushed a commit to richarddd/serverless that referenced this issue Jan 22, 2020
We were fixed on v4.2.1 due to critical bug in v4.2.2
(more info at  serverless#6667) but that's fixed with v4.2.3
astuyve pushed a commit that referenced this issue Apr 15, 2020
We were fixed on v4.2.1 due to critical bug in v4.2.2
(more info at  #6667) but that's fixed with v4.2.3
@serg06
Copy link

serg06 commented Oct 30, 2022

The issue has been reintroduced in graceful-fs 4.2.7, 16 months ago.

It's been slowing down or even crashing my builds.

I've been able to fix it by downgrading to 4.2.6, I think we should lock to that version.

@medikoo
Copy link
Contributor

medikoo commented Oct 31, 2022

@serg06 latest version of graceful-fs is v4.2.10

@serg06
Copy link

serg06 commented Oct 31, 2022

@medikoo Yes, that version has the same issue. The latest version I found without the bug is 4.2.6.

@medikoo
Copy link
Contributor

medikoo commented Oct 31, 2022

@serg06 if it would be the case, again we would have tens of reports, while nobody reported the bug since then.

@serg06
Copy link

serg06 commented Oct 31, 2022

@medikoo I was scared to make a bug report because there are already threads on this topic. I imagine there were others like me too.

Nevertheless, at least my fix is now on this thread, so anyone who hits the issue can fix it themselves.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.