Can't run vitest tests in CI

[jimohalloran]

I've run into an issue where laravel-vite-plugin is preventing vitest tests from running in a CI environment (BitBucket Pipelines in my case, but anything that sets the CI variable would do it).

• Laravel Vite Plugin Version: 0.7.0 (0.6.1 or later)
• Laravel Version: 9.36.4
• Node Version: 16.14.0
• NPM Version: 8.3.1
• Host operating system: Linux
• Web Browser & Version: N/A
• Running in Sail / Docker: Docker (CI environment)

Description:

I'm working on a pretty standard Jetstream based, Inertia/Vue application which has been set up to use Vite according to the docs in this repository. I needed some unit tests for my code, and followed https://vuejs.org/guide/scaling-up/testing.html#recipes to add vitest. That all worked fine locally, including hot reloading, and running npm test run to run tests once. When I added either npm test or npm test run to my CI server setup, my builds started failing with the following stack trace:

⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯
Error: You should not run the Vite HMR server in CI environments. You should build your assets for production instead.
    at ensureCommandShouldRunInEnvironment (/opt/atlassian/pipelines/agent/build/node_modules/laravel-vite-plugin/dist/index.js:169:15)
    at config (/opt/atlassian/pipelines/agent/build/node_modules/laravel-vite-plugin/dist/index.js:54:13)
    at runConfigHook (file:///opt/atlassian/pipelines/agent/build/node_modules/vite/dist/node/chunks/dep-4da11a5e.js:63556:31)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async resolveConfig (file:///opt/atlassian/pipelines/agent/build/node_modules/vite/dist/node/chunks/dep-4da11a5e.js:63076:14)
    at async createServer (file:///opt/atlassian/pipelines/agent/build/node_modules/vite/dist/node/chunks/dep-4da11a5e.js:62092:20)
    at async createVitest (file:///opt/atlassian/pipelines/agent/build/node_modules/vitest/dist/chunk-vite-node-externalize.72a4d20b.js:11390:18)
    at async startVitest (file:///opt/atlassian/pipelines/agent/build/node_modules/vitest/dist/chunk-vite-node-externalize.72a4d20b.js:11501:15)
    at async start (file:///opt/atlassian/pipelines/agent/build/node_modules/vitest/dist/cli.js:697:17)

Looking at the code in ensureCommandShouldRunInEnvironment it's looking to see if a production build is being run, or a bypass variable is set before it throws that error. I think it should also be checking to make sure it's not a vitest test session as well.

Steps To Reproduce:

1. Take any Laravel application with Vite and laravel-vite-plugin set up.
2. Install vitest as per https://vuejs.org/guide/scaling-up/testing.html#recipes
3. Create a simple smoke test in resources/js/__tests__/smoke.test.js:

describe('smoke test', () => {
  it('is true', () => {
    expect(true).toBeTruthy()
  })
})

5. Execute npm test run to make sure everything works. This will run the tests once and exit without starting HMR.
6. You can fake a CI environment and see the error by running CI=true npm test run.

I would expect the output from steps 5 and 5 to be the same, but 6 actually throws the stack track above as it thinks HMR is being invoked in CI. If the CI env variable is set, and you don't include "run" on the command line vitest will run the tests once and not go into watch mode, so the fix isn't quite as simple as adding "run" to the commands checked at line 156.

[timacdonald]

You can set the env LARAVEL_BYPASS_ENV_CHECK=1 to bypass the env check.

[jimohalloran]

That's absolutely the case, and does resolve the issue. But laravel-vite-plugin doesn't require setting an environment variable for production builds (because it knows HMR isn't involved). However it does block a vitest CI build because it falsely believes HMR is required. It's just not consistent.

If the solution is to require setting of the environment variable for vitest can this be documented somewhere? It took a fair bit of work to discover the environment variable "fix" today, a documentation update that helps the next person would be appreciated.

[driesvints]

We’d appreciate a pr to the docs 👍

[timacdonald]

I think we should just improve the error message to mention it. I don’t think we should document this tbh.

[timacdonald]

PR to improve the error message: #165

[jimohalloran]

Sorry for the delayed response. I think the changes to the error message are a great improvement, and addresses the concern I had. It was very confusing as a new user of vitest to be getting this error message that seemed unlreated to the issue at hand. Now that the error messaging prompts with a possible solution, hopefully that'll save the next person from burning a few hours on this issue!

Thanks everyone for getting this done and released into the world!

👍

[timacdonald]

no problem. thank you for raising the issue and the feedback ❤️