Fix reading source file multiple times

[pushplay]

Before submitting a pull request, please take a look at our
Contributing guidelines and verify:

• If you've added code that should be tested, please add tests.
• Ensure your code lints and the test suite passes (yarn lint) & (yarn test).

@sentry/node/parsers/readSourceFiles() expects the list of source files to be deduplicated (

sentry-javascript/packages/node/src/parsers.ts

Line 71 in 8870389

// we're relying on filenames being de-duped already

). If parseStack() doesn't deduplicate the list then the same file is opened multiple times. If the file is large (which can happen when using webpack) then this can waste enough memory to be a problem on a memory-constrained instance.

For example this stack:

[ { columnNumber: 11,
    fileName: '/var/task/index.js',
    functionName: null,
    lineNumber: 339944,
    methodName: null,
    native: false,
    typeName: 'Object' },
  { columnNumber: null,
    fileName: null,
    functionName: 'Generator.next',
    lineNumber: null,
    methodName: 'next',
    native: false,
    typeName: 'Generator' },
  { columnNumber: 67,
    fileName: '/var/task/index.js',
    functionName: null,
    lineNumber: 339920,
    methodName: null,
    native: false,
    typeName: null },
  { columnNumber: null,
    fileName: null,
    functionName: 'new Promise',
    lineNumber: null,
    methodName: null,
    native: false,
    typeName: null },
  { columnNumber: 10,
    fileName: '/var/task/index.js',
    functionName: 'module.exports../src/lambdas/rest/user/index.ts.__awaiter',
    lineNumber: 339899,
    methodName: '__awaiter',
    native: false,
    typeName: 'module.exports../src/lambdas/rest/user/index.ts' },
  { columnNumber: 47,
    fileName: '/var/task/index.js',
    functionName: 'Object.router.route.handler.evt [as handler]',
    lineNumber: 339943,
    methodName: 'evt [as handler]',
    native: false,
    typeName: 'Object.router.route.handler' },
  { columnNumber: 42,
    fileName: '/var/task/index.js',
    functionName: null,
    lineNumber: 13489,
    methodName: null,
    native: false,
    typeName: 'BuildableRoute' },
  { columnNumber: null,
    fileName: null,
    functionName: 'Generator.next',
    lineNumber: null,
    methodName: 'next',
    native: false,
    typeName: 'Generator' },
  { columnNumber: 67,
    fileName: '/var/task/index.js',
    functionName: null,
    lineNumber: 13429,
    methodName: null,
    native: false,
    typeName: null },
  { columnNumber: null,
    fileName: null,
    functionName: 'new Promise',
    lineNumber: null,
    methodName: null,
    native: false,
    typeName: null },
  { columnNumber: 10,
    fileName: '/var/task/index.js',
    functionName:
     'module.exports../node_modules/cassava/dist/routes/BuildableRoute.js.__awaiter',
    lineNumber: 13408,
    methodName: '__awaiter',
    native: false,
    typeName:
     'module.exports../node_modules/cassava/dist/routes/BuildableRoute.js' },
  { columnNumber: 12,
    fileName: '/var/task/index.js',
    functionName: 'BuildableRoute.handle',
    lineNumber: 13471,
    methodName: 'handle',
    native: false,
    typeName: 'BuildableRoute' },
  { columnNumber: 34,
    fileName: '/var/task/index.js',
    functionName: null,
    lineNumber: 12653,
    methodName: null,
    native: false,
    typeName: 'Router' },
  { columnNumber: null,
    fileName: null,
    functionName: 'Generator.next',
    lineNumber: null,
    methodName: 'next',
    native: false,
    typeName: 'Generator' },
  { columnNumber: 24,
    fileName: '/var/task/index.js',
    functionName: 'fulfilled',
    lineNumber: 12516,
    methodName: null,
    native: false,
    typeName: null },
  { columnNumber: 7,
    fileName: 'internal/process/next_tick.js',
    functionName: 'process._tickCallback',
    lineNumber: 68,
    methodName: '_tickCallback',
    native: false,
    typeName: 'process' } ]

generates filesToRead

[ '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js',
  '/var/task/index.js' ]

[pushplay]

The build is stuck on 1995.6 @sentry/browser - browserstack integration tests

$ node scripts/checkbrowsers.js
Unable to find API credentials in env. Please export them as BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY.

Is that something I'm responsible for?

[HazAT]

No, you are not responsible for the fail, all good.
Thank you for the PR!

I wonder why this happened, the reason is this:

https://github.com/getsentry/sentry-javascript/pull/2569/files#diff-fce4b2217c221b6d7f9e46155ca24471R88

We have a cache for the file content if we already read the file, I wonder if this is not working.
But regardless I think the change is fine anyway :)

[pushplay]

We have a cache for the file content if we already read the file, I wonder if this is not working.

There is a cache but it's only written to once the file is read. So the cache can only be hit on consecutive calls to readSourceFiles.

[rhcarvalho]

Works for me.

Wondering if filesToRead should have been a Set.

[kamilogorek]

@rhcarvalho no, as we cannot use Set due to IE10-IE11 support. They don't have Set API.