Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Jest unit tests have a memory leak or open handle that stops jest from exiting gracefully #307

Open
boktorbb opened this issue Apr 21, 2021 · 5 comments
Labels
bug Something isn't working technical debt If not paid, jeapardizes long-term success and maintainability of the repository. test:unit

Comments

@boktorbb
Copy link
Contributor

boktorbb commented Apr 21, 2021

Describe the bug

When running the jest unit tests with command yarn test:jest tests pass but we get a warning that is:

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.

This warning seems to be triggering job failures in our GitHub actions.

To Reproduce
Steps to reproduce the behavior:

  1. run yarn test:jest in the OpenSearch Dashboards directory

Expected behavior
The Jest unit tests should all pass and Jest should exit gracefully without having to pass a --forceExit flag to the command

OpenSearch Version
1.0

Dashboards Version
1.0

Additional context
I checked out Kibana 7.10.2 and they seem to have the same issue when running their jest unit tests.

@boktorbb boktorbb added bug Something isn't working test:unit labels Apr 21, 2021
@boktorbb boktorbb added this to the Beta release milestone Apr 21, 2021
@boktorbb boktorbb added this to To do in Dashboards Core via automation Apr 21, 2021
@tmarkley tmarkley removed this from the Beta release milestone Jun 30, 2021
@boktorbb
Copy link
Contributor Author

  • plugins_discovery.test.ts
  • extract_default_translations.test.js
  • download.test.js
  • pack.test.js
  • telemetry_service.test.ts
  • check_collector__integrity.test.ts
  • scan_delete.test.ts
  • config.test.ts
  • projects.test.ts
  • manage_schema.test.ts
  • decompress.test.js
  • loader.test.ts
  • integrate_locale_files.test.ts
  • project.test.ts
  • zip.test.js
  • log_rotator.test.ts
  • run.test.ts
  • projects_tree.test.ts
  • generate_build_notice_text.test.js

The above is a list of all the test files that fail because of the leaky handle with the following error message:
Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 5000 ms timeout specified by jest.setTimeout.Error:

@boktorbb
Copy link
Contributor Author

Attempted to resolve the issues on plugins_discovery.test.ts and log_rotator.test.ts by first closing a possible open handle with done() but the tests use async/await and done() does not work.

The second attempt was to increase the global jest timeout to a higher value in case the promises just take a longer period of time to resolve. I tried raising the value from the default 5000ms up to 10,000ms, and 30,000ms. Neither attempt managed to solve the timeout issue.*

The third attempt was to clear and reset the mocks in the unit tests in case they had an effect on the promises resolving properly. In the afterEach() function for the tests I added jest.clearAllMocks() and jest.resetAllMocks() but no luck.

  • Increasing the timeout did not seem to have an effect on plugins_discover.test.ts and log_rotator.test.ts but seems to solve the issue for some of the other leaky tests

@boktorbb
Copy link
Contributor Author

    Test Suites: 19 failed, 3 skipped, 1413 passed, 1432 of 1435 total
    Tests:       98 failed, 41 skipped, 9 todo, 10518 passed, 10666 total
    Snapshots:   2383 passed, 2383 total
    Time:        1507.986 s
    Ran all test suites.
     
    Jest has detected the following 41 open handles potentially keeping Jest from exiting:
     
      ●  FSREQWRAP
     
          119 |             })
          120 |             .get('/plugin.zip')
        > 121 |             .replyWithFile(200, filePath);
              |              ^
          122 |
          123 |           const sourceUrl = 'http://example.com/plugin.zip';
          124 |
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:121:14)
     
     
      ●  FSREQWRAP
     
          39 |     const fileInfo = statSync(sourcePath);
          40 |
        > 41 |     const readStream = createReadStream(sourcePath);
             |                        ^
          42 |
          43 |     return { readStream, fileInfo };
          44 |   } catch (err) {
     
          at openSourceFile (src/cli_plugin/install/downloaders/file.js:41:24)
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:77:38)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:146:18)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:146:18)
     
     
      ●  FSREQWRAP
     
          202 |           .reply(404)
          203 |           .get('/goodfile.tar.gz')
        > 204 |           .replyWithFile(200, filePath);
              |            ^
          205 |
          206 |         return download(settings, logger).then(function () {
          207 |           expect(logger.log.getCall(0).args[0]).toMatch(/badfile1.tar.gz/);
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:204:12)
     
     
      ●  FSREQWRAP
     
          231 |           .reply(404)
          232 |           .get('/goodfile.tar.gz')
        > 233 |           .replyWithFile(200, filePath)
              |            ^
          234 |           .get('/badfile3.tar.gz')
          235 |           .reply(404);
          236 |
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:233:12)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:360:9)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:369:9)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:379:9)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:389:9)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:399:9)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:409:9)
     
     
      ●  FSREQWRAP
     
          312 |         nock(url)
          313 |           .get('/plugin.zip')
        > 314 |           .replyWithFile(200, join(__dirname, '__fixtures__/replies/test_plugin.zip'));
              |            ^
          315 |       }
          316 |
          317 |       beforeAll(function (done) {
     
          at Interceptor.replyWithFile (node_modules/nock/lib/interceptor.js:197:27)
          at nockPluginForUrl (src/cli_plugin/install/download.test.js:314:12)
          at Object.<anonymous> (src/cli_plugin/install/download.test.js:419:9)
     
     
      ●  FSREQWRAP
     
          43 | async function parseJsonFile(relativePath: string) {
          44 |   const schemaPath = path.resolve(__dirname, '__fixture__', relativePath);
        > 45 |   const fileContent = await read(schemaPath, 'utf8');
             |                             ^
          46 |   return JSON.parse(fileContent);
          47 | }
          48 |
     
          at parseJsonFile (packages/osd-telemetry-tools/src/tools/check_collector__integrity.test.ts:45:29)
          at Object.it (packages/osd-telemetry-tools/src/tools/check_collector__integrity.test.ts:51:30)
     
     
      ●  FSREQWRAP
     
          43 | async function parseJsonFile(relativePath: string) {
          44 |   const schemaPath = path.resolve(__dirname, '__fixture__', relativePath);
        > 45 |   const fileContent = await read(schemaPath, 'utf8');
             |                             ^
          46 |   return JSON.parse(fileContent);
          47 | }
          48 |
     
          at parseJsonFile (packages/osd-telemetry-tools/src/tools/check_collector__integrity.test.ts:45:29)
          at Object.it (packages/osd-telemetry-tools/src/tools/check_collector__integrity.test.ts:58:32)
     
     
      ●  FSREQWRAP
     
          43 | async function parseJsonFile(relativePath: string) {
          44 |   const schemaPath = path.resolve(__dirname, '__fixture__', relativePath);
        > 45 |   const fileContent = await read(schemaPath, 'utf8');
             |                             ^
          46 |   return JSON.parse(fileContent);
          47 | }
          48 |
     
          at parseJsonFile (packages/osd-telemetry-tools/src/tools/check_collector__integrity.test.ts:45:29)
          at Object.it (packages/osd-telemetry-tools/src/tools/check_collector__integrity.test.ts:74:32)
     
     
      ●  FSREQWRAP
     
          212 |   options: IntegrateOptions
          213 | ) {
        > 214 |   const localizedMessages = JSON.parse((await readFileAsync(options.sourceFileName)).toString());
              |                                               ^
          215 |   if (!localizedMessages.formats) {
          216 |     throw createFailError(`Locale file should contain formats object.`);
          217 |   }
     
          at integrateLocaleFiles (src/dev/i18n/integrate_locale_files.ts:214:47)
          at Object.test (src/dev/i18n/integrate_locale_files.test.ts:180:13)
     
     
      ●  FSREQWRAP
     
          52 | describe('#getProjects', () => {
          53 |   beforeAll(async () => {
        > 54 |     await promisify(mkdir)(rootPlugins);
             |           ^
          55 |
          56 |     await promisify(symlink)(
          57 |       join(__dirname, '__fixtures__/symlinked-plugins/corge'),
     
          at Object.beforeAll (packages/osd-pm/src/utils/projects.test.ts:54:11)
     
     
      ●  FSREQWRAP
     
          47 |
          48 |   const rcFile = path.resolve(rcRoot, TELEMETRY_RC);
        > 49 |   const configString = await readFileAsync(rcFile, 'utf8');
             |                              ^
          50 |   return JSON.parse(configString);
          51 | }
          52 |
     
          at readRcFile (packages/osd-telemetry-tools/src/tools/config.ts:49:30)
          at parseTelemetryRC (packages/osd-telemetry-tools/src/tools/config.ts:54:26)
          at Object.it (packages/osd-telemetry-tools/src/tools/config.test.ts:44:26)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:94:15)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:118:15)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:132:15)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:146:15)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:170:15)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:177:15)
     
     
      ●  FSREQWRAP
     
          76 |   try {
          77 |     const { readStream, fileInfo } = openSourceFile({ sourcePath });
        > 78 |     const writeStream = createWriteStream(targetPath);
             |                         ^
          79 |
          80 |     try {
          81 |       const progress = new Progress(logger);
     
          at downloadLocalFile (src/cli_plugin/install/downloaders/file.js:78:25)
          at _downloadSingle (src/cli_plugin/install/download.js:66:23)
          at copyReplyFile (src/cli_plugin/install/pack.test.js:87:14)
          at Object.it (src/cli_plugin/install/pack.test.js:184:15)
     
     
      ●  FSREQWRAP
     
          40 | async function parseJsonFile(relativePath: string) {
          41 |   const schemaPath = path.resolve(__dirname, '__fixture__', relativePath);
        > 42 |   const fileContent = await read(schemaPath, 'utf8');
             |                             ^
          43 |   return JSON.parse(fileContent);
          44 | }
          45 |
     
          at parseJsonFile (packages/osd-telemetry-tools/src/tools/manage_schema.test.ts:42:29)
          at Object.it (packages/osd-telemetry-tools/src/tools/manage_schema.test.ts:48:30)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:87:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:102:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:124:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:146:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:174:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:213:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:242:22)
     
     
      ●  FSREQWRAP
     
          227 |   async _getLogFileSizeAndCreateIfNeeded() {
          228 |     try {
        > 229 |       const logFileStats = await statAsync(this.logFilePath);
              |                                  ^
          230 |       return logFileStats.size;
          231 |     } catch {
          232 |       // touch the file to make the watcher being able to register
     
          at LogRotator._getLogFileSizeAndCreateIfNeeded (src/legacy/server/logging/rotate/log_rotator.ts:229:34)
          at LogRotator._callRotateOnStartup (src/legacy/server/logging/rotate/log_rotator.ts:240:35)
          at LogRotator.start (src/legacy/server/logging/rotate/log_rotator.ts:96:16)
          at Object.it (src/legacy/server/logging/rotate/log_rotator.test.ts:269:22)
     
     
      ●  FSREQWRAP
     
          53 |   fs.mkdirSync(output, { recursive: true });
          54 |   return new Promise((resolve, reject) => {
        > 55 |     yauzl.open(input, { lazyEntries: true }, (err, zipfile) => {
             |           ^
          56 |       if (err) {
          57 |         reject(err);
          58 |       }
     
          at Object.open (node_modules/yauzl/index.js:33:6)
          at Promise (packages/osd-opensearch/src/utils/decompress.js:55:11)
          at decompressZip (packages/osd-opensearch/src/utils/decompress.js:54:10)
          at decompress (packages/osd-opensearch/src/utils/decompress.js:101:13)
          at Object.<anonymous>.test (packages/osd-opensearch/src/utils/decompress.test.js:62:9)
     
     
      ●  ZLIB
     
          42 |     fs.createReadStream(archive)
          43 |       .on('error', reject)
        > 44 |       .pipe(zlib.createGunzip())
             |                  ^
          45 |       .on('error', reject)
          46 |       .pipe(tarFs.extract(dirPath, { strip: true }))
          47 |       .on('error', reject)
     
          at Promise (packages/osd-opensearch/src/utils/decompress.js:44:18)
          at decompressTarball (packages/osd-opensearch/src/utils/decompress.js:41:10)
          at decompress (packages/osd-opensearch/src/utils/decompress.js:105:13)
          at Object.<anonymous>.test (packages/osd-opensearch/src/utils/decompress.test.js:67:9)
     
     
      ●  FSREQWRAP
     
          45 |
          46 | export function readPackageJson(cwd: string): IPackageJson {
        > 47 |   return readPkg({ cwd, normalize: false });
             |          ^
          48 | }
          49 |
          50 | export function writePackageJson(path: string, json: IPackageJson) {
     
          at Object.<anonymous>.module.exports (node_modules/read-pkg/index.js:17:31)
          at readPackageJson (packages/osd-pm/src/utils/package_json.ts:47:10)
          at Function.fromPath (packages/osd-pm/src/utils/project.ts:65:27)
          at Object.test (packages/osd-pm/src/utils/project.test.ts:52:48)
     
     
      ●  FSREQWRAP
     
          54 |
          55 |   return new Promise((resolve, reject) => {
        > 56 |     yauzl.open(archive, { lazyEntries: true }, function (err, zipfile) {
             |           ^
          57 |       if (err) {
          58 |         return reject(err);
          59 |       }
     
          at Object.open (node_modules/yauzl/index.js:33:6)
          at Promise (src/cli_plugin/install/zip.js:56:11)
          at analyzeArchive (src/cli_plugin/install/zip.js:55:10)
          at Object.it (src/cli_plugin/install/zip.test.js:59:32)
     
     
      ●  FSREQWRAP
     
          109 | export function extractArchive(archive, targetDir, stripPrefix) {
          110 |   return new Promise((resolve, reject) => {
        > 111 |     yauzl.open(archive, { lazyEntries: true }, function (err, zipfile) {
              |           ^
          112 |       if (err) {
          113 |         return reject(err);
          114 |       }
     
          at Object.open (node_modules/yauzl/index.js:33:6)
          at Promise (src/cli_plugin/install/zip.js:111:11)
          at extractArchive (src/cli_plugin/install/zip.js:110:10)
          at Object.it (src/cli_plugin/install/zip.test.js:75:15)
     
     
      ●  FSREQWRAP
     
          109 | export function extractArchive(archive, targetDir, stripPrefix) {
          110 |   return new Promise((resolve, reject) => {
        > 111 |     yauzl.open(archive, { lazyEntries: true }, function (err, zipfile) {
              |           ^
          112 |       if (err) {
          113 |         return reject(err);
          114 |       }
     
          at Object.open (node_modules/yauzl/index.js:33:6)
          at Promise (src/cli_plugin/install/zip.js:111:11)
          at extractArchive (src/cli_plugin/install/zip.js:110:10)
          at Object.it (src/cli_plugin/install/zip.test.js:96:9)
     
     
      ●  TTYWRAP
     
          43 | export function confirm(question, options = {}) {
          44 |   const rl = createInterface({
        > 45 |     input: options.input || process.stdin,
             |                                     ^
          46 |     output: options.output || process.stdout,
          47 |   });
          48 |
     
          at confirm (src/cli_keystore/utils/prompt.js:45:37)
          at Object.it (src/cli_keystore/utils/prompt.test.js:57:7)

@boktorbb
Copy link
Contributor Author

boktorbb commented Sep 1, 2021

Unit tests when run with --detectOpenHandles arg after they have already been run tend to crash and give the following error:

<--- Last few GCs --->

[49272:0x3dc5960]  1770050 ms: Scavenge 1283.2 (1421.6) -> 1282.7 (1422.1) MB, 2.6 / 0.0 ms  (average mu = 0.197, current mu = 0.213) allocation failure
[49272:0x3dc5960]  1770062 ms: Scavenge 1283.6 (1422.1) -> 1283.3 (1422.6) MB, 2.7 / 0.0 ms  (average mu = 0.197, current mu = 0.213) allocation failure
[49272:0x3dc5960]  1770071 ms: Scavenge 1284.0 (1422.6) -> 1283.8 (1423.6) MB, 2.3 / 0.0 ms  (average mu = 0.197, current mu = 0.213) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x256a7695bf1d]
    1: ConstructFrame [pc: 0x256a7690d145]
    2: StubFrame [pc: 0x256a76989d9d]
Security context: 0x164ac361e6c1 <JSObject>
    3: new Script(aka Script) [0x525669512f1] [vm.js:83] [bytecode=0xd058f56f0c1 offset=373](this=0x3d76b9802801 <the_hole>,/* anonymous */=0x33b61b237089 <Very long string[142113]>,/* anonymous */=0x33b61b2370b1 <Object map = 0x28346d7ffea9>)
    4: ConstructFrame [pc: 0x256a76...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8fb090 node::Abort() [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 2: 0x8fb0dc  [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 3: 0xb0336e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 4: 0xb035a4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 5: 0xef7602  [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 6: 0xef7708 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 7: 0xf037e2 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 8: 0xf04114 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
 9: 0xf06d81 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
10: 0xed69cb v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
11: 0x1020253 v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
12: 0x1092669 v8::internal::parsing::ParseProgram(v8::internal::ParseInfo*, v8::internal::Isolate*) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
13: 0xc436cb  [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
14: 0xc4557a v8::internal::Compiler::GetSharedFunctionInfoForScript(v8::internal::Handle<v8::internal::String>, v8::internal::Compiler::ScriptDetails const&, v8::ScriptOriginOptions, v8::Extension*, v8::internal::ScriptData*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason, v8::internal::NativesFlag) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
15: 0xb2aa48 v8::ScriptCompiler::CompileUnboundInternal(v8::Isolate*, v8::ScriptCompiler::Source*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
16: 0xb2ae4d v8::ScriptCompiler::CompileUnboundScript(v8::Isolate*, v8::ScriptCompiler::Source*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
17: 0x92fe88 node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
18: 0xb903db  [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
19: 0xb92372 v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [/home/ubuntu/.nvm/versions/node/v10.24.1/bin/node]
20: 0x256a7695bf1d
Aborted (core dumped)

@boktorbb
Copy link
Contributor Author

boktorbb commented Sep 1, 2021

Hooking into the tests with the vscode debugger gives a look into the stacktrace before the out of memory exception:

Screen Shot 2021-09-01 at 2 02 31 PM

Screen Shot 2021-09-01 at 2 03 20 PM

@tmarkley tmarkley added the technical debt If not paid, jeapardizes long-term success and maintainability of the repository. label May 25, 2022
SuZhou-Joe added a commit to SuZhou-Joe/OpenSearch-Dashboards that referenced this issue Mar 22, 2024
Signed-off-by: SuZhou-Joe <suzhou@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working technical debt If not paid, jeapardizes long-term success and maintainability of the repository. test:unit
Projects
Development

No branches or pull requests

2 participants