Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typescript Usage Results in Out of Memory Exception #2316

Closed
TLadd opened this Issue Aug 10, 2018 · 8 comments

Comments

5 participants
@TLadd
Copy link
Contributor

TLadd commented Aug 10, 2018

Current behavior:

When I try to use Typescript with Cypress, Cypress Helper's memory usage quickly scales up with the number of test files. The actual number of tests does not seem to be particularly important (i.e. it's fine to have 1 file with 100 tests in it, but 50 test files with 1 test each will break). After Cypress Helper's memory usage gets to a little of 2 GB in Activity Monitor, the tests grind to a halt and eventually result in an Out of Memory exception.

Desired behavior:

When using Typescript, the memory usage remains similar to how it behaves when not using typescript.

Steps to reproduce:

I have a repo setup that illustrates the issue: https://github.com/TLadd/cypress-ts-memory-leak/tree/master

It's a repo that only contains a basic cypress setup with typescript and @cypress/webpack-preprocessor. It contains 50 test files, each with one test each expect(1).to.eq(1);. For me, running cypress run consistently results in an out of memory exception trying to run the 37th test.

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running: test42.ts...                                                                  (37 of 50)

<--- Last few GCs --->

[21977:0x7f94ab01b200]   151789 ms: Mark-sweep 2059.6 (2165.2) -> 2059.6 (2149.2) MB, 4886.1 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 4886 ms) last resort
[21977:0x7f94ab01b200]   156632 ms: Mark-sweep 2059.6 (2149.2) -> 2059.6 (2149.2) MB, 4842.6 / 0.0 ms  last resort



<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2dfba37ad681 <JSObject>
    1: set [native collection.js:~247] [pc=0x928742bfdb4](this=0x28257ee942c9 <Map map = 0x44554988609>,p=0x1f93fc19e4f1 <String[13]: MSDescription>,x=0x1587c7882bf1 <SymbolObject map = 0x1f792a0791b1>)
    3: /* anonymous */(aka /* anonymous */) [/Users/thomasladd/github/cypress-ts-memory-leak/node_modules/typescript/lib/typescript.js:30020] [bytecode=0x3ff4e4d20bf9 offset=65](this=0x34f120f023...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 2: node::FatalError(char const*, char const*) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 3: v8::internal::FatalProcessOutOfMemory(char const*) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 4: v8::internal::FatalProcessOutOfMemory(char const*) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 6: v8::internal::compiler::Node::Uses::empty() const [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 7: v8::internal::RegisterConfiguration::AreAliases(v8::internal::MachineRepresentation, int, v8::internal::MachineRepresentation, int) const [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 8: 0x92873d043fd
 9: 0x928742bfdb4
10: 0x92873df78ea
11: 0x92873dbbe1c

Versions

Cypress: 3.0.3
OS: MAC High Sierra 10.13.4
Browser: Default Electron or Chrome

@brian-mann

This comment has been minimized.

Copy link
Member

brian-mann commented Aug 11, 2018

I was able to reproduce this, but I don't believe this is a Cypress specific issue.

The memory did not increase when I renamed all of the files to .js...

  • and used our default browserify preprocessor
  • or used the webpack preprocessor with your options

This problem only exhibits itself when using ts-loader.

Perhaps ts-loader is buffering every file in memory which is why you see the escalating memory usage. I did a cursory look around how we handle the pluginsFile, and it doesn't look like we're doing anything wrong. We keep the child process open for the entire duration of the run.

There may be some options in ts-loader that ask it to write its buffer to disk as opposed to memory, perhaps that would fix it.

There may also be webpack specific options you can pass to prevent this from happening as well.

Maybe someone on our team can look at this, but it seems this may not be an issue with Cypress.

@TLadd

This comment has been minimized.

Copy link
Contributor Author

TLadd commented Aug 12, 2018

Thanks so much for looking into this. One thing that I've found that works is to pass the transpileOnly option to ts-loader.

{
  loader: "ts-loader",
  options: {
    transpileOnly: true
  }
}

Whatever ts-loader is hanging onto across tests, it doesn't do it when in transpileOnly mode; the memory used by Cypress Helper goes up and down as each test completes and starts up, always staying in the range of a couple hundred MB.

@bahmutov

This comment has been minimized.

Copy link
Collaborator

bahmutov commented Aug 12, 2018

@TLadd

This comment has been minimized.

Copy link
Contributor Author

TLadd commented Sep 26, 2018

Just an update on this, I tried updating to the latest ts-loader (v5.2.1), and even with the transpileOnly flag turned on, I end up running out of memory. Reverted back to 4.4.2 and not having issues.

@masonmark

This comment has been minimized.

Copy link

masonmark commented Oct 22, 2018

I was debugging why a seemingly-random set of our specs were failing with ERR_IPC_CHANNEL_CLOSED errors. Disabling parallelization (not passing --parallel) and running the tests sequentially made it obvious that we were running out of memory at the same spot, right after the 28th spec file, both on CI (Ubuntu 16.04) locally (macOS 10.14).

We are using ts-loader 4.3.0 with Cypress 3.1.0, and the workaround described by @TLadd above indeed worked for us.

Since we are using the "TypeScript with WebPack" example recipe as-is, this just meant adding the option transpileOnly to the webpackOptions structure inside our cypress/plugins/index.js file, like this:

      use: [{
        loader: 'ts-loader',
        options: {
          transpileOnly: true
        }
      }]
@masonmark

This comment was marked as outdated.

Copy link

masonmark commented Oct 22, 2018

I was debugging why a seemingly-random set of our specs were failing with ERR_IPC_CHANNEL_CLOSED errors. Disabling parallelization (not passing --parallel) and running the tests sequentially made it obvious that we were running out of memory at the same spot, right after the 28th spec file, both on CI (Ubuntu 16.04) locally (macOS 10.14).

We are using ts-loader 4.3.0 with Cypress 3.1.0, and the workaround described by @TLadd above indeed worked for us.

Since we are using the "TypeScript with WebPack" example recipe as-is, this just meant adding the option transpileOnly to the webpackOptions structure inside our cypress/plugins/index.js file, like this:

      use: [{
        loader: 'ts-loader',
        options: {
          transpileOnly: true
        }
      }]
@masonmark

This comment was marked as outdated.

Copy link

masonmark commented Oct 22, 2018

I was debugging why a seemingly-random set of our specs were failing with ERR_IPC_CHANNEL_CLOSED errors. Disabling parallelization (not passing --parallel) and running the tests sequentially showed that we were running out of memory at the same spot, right after the 28th spec file, both on CI (Ubuntu 16.04) locally (macOS 10.14).

We are using ts-loader 4.3.0 with Cypress 3.1.0, and the workaround described by @TLadd above indeed worked for us.

Since we are using the "TypeScript with WebPack" example recipe as-is, this just meant adding the option transpileOnly to the webpackOptions structure inside our cypress/plugins/index.js file, like this:

      use: [{
        loader: 'ts-loader',
        options: {
          transpileOnly: true
        }
      }]
@jennifer-shehane

This comment has been minimized.

Copy link
Member

jennifer-shehane commented Nov 10, 2018

More specifically, ts-loader does have a section in their docs for optimizing faster builds here: https://github.com/TypeStrong/ts-loader#faster-builds

hackergil added a commit to SAP/cloud-commerce-spartacus-storefront that referenced this issue Mar 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.