Bug: An error occurs when invoking ESLint#lintFiles() in Jest

[StewEucen]

Environment

Node version: 20.12.2
npm version: 10.5.0
Local ESLint version: 9.1.1
Global ESLint version: 9.1.1
Operating System: macOS

What parser are you using?

Default (Espree)

What did you do?

Configuration

• eslint.config.js

  'use strict'
  
  module.exports = [
    {
      languageOptions: {
        parserOptions: {
          ecmaVersion: 'latest',
        },
      },
      ignores: [
        '**/node_modules/**',
      ],
      rules: {
        indent: ['error', 2],
        quotes: ['error', 'single'],
        semi: ['error', 'never'],
      },
    },
  ]

• Actual code

  'use strict'
  
  const {
    ESLint,
  } = require('eslint')
  
  describe('ESLint Bug', () => {
    test('should not throw an error', async () => {
      const eslint = new ESLint()
      const filePaths = [
        'tests/resources',
      ]
      const expectedLength = 3
  
      const results = await eslint.lintFiles(filePaths)
  
      expect(results)
        .toHaveLength(expectedLength)
    })
  })

• Test Resources

  • tests/resources/indent.js

    'use strict'
    
    module.exports = function doubleValue (
      value,
      ignore
    ) {
      if (ignore) {
        return value
        }
    
      return value + value
    }

  • tests/resources/quotes.js

    'use strict'
    
    module.exports = "quotes"

  • tests/resources/semi.js

    'use strict'
    
    global.semi = 111;

What did you expect to happen?

• I expect that Jest shows the log below

   PASS  tests/__tests__/eslint-bug.js
    ESLint Bug
      ✓ should not throw an error (121 ms)
  
  Test Suites: 1 passed, 1 total
  Tests:       1 passed, 1 total
  Snapshots:   0 total
  Time:        1.221 s, estimated 2 s
  Ran all test suites.

What actually happened?

• An error Result is not a promise. is thrown.

   FAIL  tests/__tests__/eslint-bug.js
    ESLint Bug
      ✕ should not throw an error (105 ms)
  
    ● ESLint Bug › should not throw an error
  
      Result is not a promise.
  
        13 |     const expectedLength = 3
        14 |
      > 15 |     const results = await eslint.lintFiles(filePaths)
           |                     ^
        16 |
        17 |     expect(results)
        18 |       .toHaveLength(expectedLength)
  
        at Retrier.retry (node_modules/@humanwhocodes/retry/dist/retrier.cjs:200:35)
            at Array.map (<anonymous>)
        at Object.<anonymous> (tests/__tests__/eslint-bug.js:15:21)
  
  Test Suites: 1 failed, 1 total
  Tests:       1 failed, 1 total
  Snapshots:   0 total
  Time:        1.195 s
  Ran all test suites.

Link to Minimal Reproducible Example

https://github.com/StewEucen/eslint-bug-with-jest

Participation

• I am willing to submit a pull request for this issue.

Additional comments

• I wrote details as the problem in README.md of this repository
  https://github.com/StewEucen/eslint-bug-with-jest

  • Background
  • Steps to Reproduce
  • Cause
  • Mechanism
  • How to Fix
  • Result
  • Side Effect
  • Note

[mdjermanovic]

I can reproduce this locally. Thanks for the detailed analysis of the cause!

https://github.com/StewEucen/eslint-bug-with-jest?tab=readme-ov-file#cause

• Retrier#retry() uses Promise declared in lib.es2015.iterable.d.ts
• fs.readFile() uses Promise declared in lib.es5.d.ts

So it seems that in Node.js different operations may use different Promise constructors?

@nzakas perhaps @humanwhocodes/retry could check if the result is a thenable instead of checking result instanceof Promise?

[StewEucen]

@mdjermanovic

• Thank you for your confirmation :)

---

So it seems that in Node.js different operations may use different Promise constructors?

• I think that since fs is old package, it uses polyfill Promise class.

---

• I have already found two ways to fix now.

  1. Change argument of #retry() function to async in ESLint repository.

     -                return retrier.retry(() => fs.readFile(...)
     +                return retrier.retry(async () => fs.readFile(...)

  2. Change the logic to confirm Promise in #retry() of @humanwhocodes/retry repository.

     -        if (!(result instanceof Promise)) {
     +        if (
     +          !(result instanceof Promise)
     +          && (!result || typeof result.then !== 'function')
     +        ) {

• I think that 2. is a fixing ad-hoc, thus we can select 1.

[fasttime]

The Promise object returned by fs.readFile is from a different realm:

This only occurs when the Node.js process is launched with the --experimental-vm-modules flag. It may have something to do with Jest's experimental ESM support, which is enabled by the flag.

[StewEucen]

@fasttime

• Thank you for your information

---

• I used --experimental-vm-modules on run Jest, it was due to the following reasons.

1. I had used ESLint#lintFiles() in Jest with ESLint@8.x.x. As we know, Jest works as CommonJS. But It did not require to use --experimental-vm-modules flag.

2. I changed the code to fit Flat Config with 9.1.1, but it was not work. Because new code of ESLint#lintFiles() uses dynamic import.

   https://github.com/eslint/eslint/blob/v9.1.1/lib/eslint/eslint.js#L317

   async function loadFlatConfigFile(filePath) {
       ...
   
       const config = (await import(fileURL)).default;
   
       ...
   }

   https://github.com/eslint/eslint/blob/v9.1.1/lib/eslint/eslint.js#L369

   async function calculateConfigArray(eslint, { ... }) {
       ...
   
       if (configFilePath) {
           const fileConfig = await loadFlatConfigFile(configFilePath);
   
           ...
       }
   
       ...
   }

   https://github.com/eslint/eslint/blob/v9.1.1/lib/eslint/eslint.js#L815

   class ESLint {
       ...
   
       async lintFiles(patterns) {
           ...
   
           const configs = await calculateConfigArray(this, eslintOptions);
   
           ...
       }
   
       ...
   }

3. To solve the problem, I added a --experimental-vm-modules flag for Jest script.

   https://github.com/StewEucen/eslint-bug-with-jest/blob/main/package.json#L7

   {
     "scripts": {
       "test": "export NODE_OPTIONS=\"$NODE_OPTIONS --experimental-vm-modules\"; jest --forceExit"
     },
   }

4. And then, I got the behavior that I reported in this issue.

[nzakas]

I can update retry. Never occurred to me that there might be a promise from another realm involved.

[nzakas]

A new version of @humanwhocodes/retry was released as a patch version that should fix this.

[mdjermanovic]

Here's a PR to update the dependency: #18416

[StewEucen]

@nzakas @mdjermanovic @fasttime

• Thank you for your work related to this issue.
• I am going to use ESLint patched version for my repository.