[🐛 Bug]: Better resolve dynamic imports in browser tests

[christian-bromann]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

WebdriverIO Version

latest

Node.js Version

latest

Mode

WDIO Testrunner

Which capabilities are you using?

n/a

What happened?

When running browser tests and importing CJS modules through Vite, it sometimes happen that we get the following error:

Error:  Test failed due to following error(s):
  - pr-eact.test.tsx: Failed to fetch dynamically imported module: http://localhost:39259/node_modules/.vite/deps/@testing-library_preact.js?v=60c2fbb0: TypeError: Failed to fetch dynamically imported module: http://localhost:39259/node_modules/.vite/deps/@testing-library_preact.js?v=60c2fbb0

Usually it resolves when reloading the page which is why we look for this error in the browser runner. However we have experienced in our build that sometimes even reloading doesn't resolve the issue.

What is your expected behavior?

The CJS transformation should work flawlessly, the error should not appear.

How to reproduce the bug.

Unskip this test and run it in CI.

Relevant log output

- pr-eact.test.tsx: Failed to fetch dynamically imported module: http://localhost:39259/node_modules/.vite/deps/@testing-library_preact.js?v=60c2fbb0: TypeError: Failed to fetch dynamically imported module: http://localhost:39259/node_modules/.vite/deps/@testing-library_preact.js?v=60c2fbb0

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

[wdio-bot]

Thanks for reporting!

We greatly appreciate any contributions that help resolve the bug. While we understand that active contributors have their own priorities, we kindly request your assistance if you rely on this bug being fixed. We encourage you to take a look at our contribution guidelines or join our friendly Discord development server, where you can ask any questions you may have. Thank you for your support, and cheers!

[nzakas]

Looks like our browser tests are passing again.

Because it seems like our tests fail frequently after a new wdio release, would it be possible for you to add an ESLint integration test as part of your release CI?

[christian-bromann]

@nzakas yes, I can look into that!

[nzakas]

@christian-bromann thank you 🙏

[RoXuS]

hey!

We experiment the same randomly errors, it happens only on gitlab (with a standalone chrome) AND on random tests.
I think it link to the CPU/memory because on our Mac ARM we don't have seen this error.

We use "lit" preset and we are migrating all our 1400 tests to webdriverio.

But we can't continue our migration, our pipelines doesn't pass anymore...
It seems to happen with tests using some external libraries like sinon or fetch-mock.

It started when we added these 2 imports:

import fetchMock from 'fetch-mock/esm/client';
import sinon from 'sinon';

The error:

errors: [
[0-3]     {
[0-3]       filename: '/builds/insight/frontend-monorepo/monorepo/packages/libs/business-components/dt-contract-page/test/dt-contract-page-custom-fields/dt-contract-page-custom-fields.wdio.ts',
[0-3]       message: 'Failed to fetch dynamically imported module: http://localhost:42635/node_modules/.vite/deps/fetch-mock_esm_client.js?v=0d644ae5: TypeError: Failed to fetch dynamically imported module: [http://localhost:42635/node_modules/.vite/deps/fetch-mock_esm_client.js?v=0d644ae5'](http://localhost:42635/node_modules/.vite/deps/fetch-mock_esm_client.js?v=0d644ae5%27)
[0-3]     }
[0-3]   ],

EDIT : to provide clarification, it also happens on our macs but very rarely!

EDIT2: the runner retry the test but without success

[0-3] 2024-02-05T09:05:53.290Z INFO @wdio/runner: Retry test run due to dynamic import error

EDIT3: this never happens when I run only one test, each time it's when I run the entire test suite, if I set the maxInstances to 1, it never happens too.

[christian-bromann]

@RoXuS thanks, this is very interesting. This issue is on top of my list so hopefully I have a solution to this soon to unblock you.

We use "lit" preset and we are migrating all our 1400 tests to webdriverio.

That's awesome!

[swendlandt]

Hello @christian-bromann,

did you have a chance to look into this issue already? We are facing the same issue and our pipeline is often blocked by this.

[RoXuS]

yeah it is really frustrating, to avoid this issue we have to set maxInstances to 1 in case of CI:

maxInstances: process.env.CI ? 1 : 6

It is ok but execution take 11 minutes instead of 4 minutes...