Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible memory-use regression in v1.48.1 #526

Open
mshick opened this issue Feb 13, 2024 · 1 comment
Open

Possible memory-use regression in v1.48.1 #526

mshick opened this issue Feb 13, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@mshick
Copy link

mshick commented Feb 13, 2024

Describe the bug
There appears to be a memory use regression in v1.48.1, which is presumably a result of the disposeContext behavior here.

When attempting to run serverless package locally there is a pause of several minutes on a fairly large serverless project with around 30 fns. After the pause bundles start being emitted and the build continues as normal. This issue does not exist on the immediately preceding version, v1.48.0 or any version tested before that, back to v1.43.0. Multiple version of esbuild between v0.17.3 and v0.19.12 have been tested, and multiple relatively recent versions of serverless

More important than the performance regression is that during this pause, in GitHub Actions CI on a custom 4 CPU runner the process fails entirely, with what is clearly an OOM.

Error:
  Error: The service was stopped
      at /home/runner/work/takeshape/takeshape/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1100:28
      at responseCallbacks.<computed> (/home/runner/work/takeshape/takeshape/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:697:9)
      at Socket.afterClose (/home/runner/work/takeshape/takeshape/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:687:28)
      at Socket.emit (node:events:530:35)
      at Socket.emit (node:domain:488:12)
      at endReadableNT (node:internal/streams/readable:1696:12)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

To Reproduce

  1. Create a large project, with 100s of dependencies including aws-sdk, etc., being bundled in.
  2. Install v1.48.1 or later of this plugin.
  3. Configure this plugin to logLevel: info.
  4. Run serverless package.
  5. Note the extended wait before bundles are emitted.
  6. Attempt to run in GitHub Actions. Note that the process will fail with little info.

Expected behavior
On earlier versions there is not the extended wait, nor does this fail in CI.

Versions (please complete the following information):

  • OS: macOS 14.3.1 (M1 Pro), GitHub Actions Ubuntu 4CPU runner
  • Serverless Framework Version: 3.38.0 (and others)
  • Plugin Version: 1.48.1 (and later)

Additional context

Have also tried different concurrency settings (1, 2, 3, 4) with no change.

Here is a pared down config that has the issue for me:

  const common = {
    logLevel: 'info',
    packager: 'pnpm',
    concurrency: 1,
    zipConcurrency: 1,
    write: false,
    minify: true,
    treeShaking: true,
    define: {
      'process.env.NODE_ENV': '"production"'
    },
    exclude: []
  };
@mshick mshick added the bug Something isn't working label Feb 13, 2024
@BorjaMacedo
Copy link
Contributor

BorjaMacedo commented Feb 21, 2024

When apis are very large, a memory leak occurs due to context.rebuild().
I have a proposal here #528 that seems to solve the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants