dev server race condition in optimizeDeps with noDiscovery: true

[kherock]

Describe the bug

When running the Vite dev server, I believe that there's an issue with the timing of the promise returned by ViteDevServer.listen() when the optimizer is enabled and noDiscovery is set to true. With this configuration, Vite can throw an error when ViteDevServer.transformRequest() is called immediately after the server reports that it is listening.

ENOENT: no such file or directory, open '<path>/node_modules/.vitest/deps_temp_0722f042/_metadata.json'

The issue is that Vite will commit dependencies twice for the initial optimization run - once for the run created the moment that the server starts, and again for the first request it handles (if it occurs before the run finishes).

This error is very timing sensitive, and doesn't occur if Vite is initially busy processing requests that do not require the optimizer. For example, I have a Vitest suite that processes a few files in SSR mode, and then it moves onto files that use the dependency optimizer in web mode. The error only occurs when I skip the files that Vite transforms with the SSR configuration.

---

I have 2 fixes for this issue, and I believe this is the offending section of code:

vite/packages/vite/src/node/optimizer/optimizer.ts

Lines 220 to 226 in 45c6f3b

	if (config.optimizeDeps.noDiscovery) {
	// We don't need to scan for dependencies or wait for the static crawl to end
	// Run the first optimization run immediately
	runOptimizer()
	} else if (!isBuild) {
	// Important, the scanner is dev only
	depsOptimizer.scanProcessing = new Promise((resolve) => {

If I use await runOptimizer() on line 223, the issue goes away. I can also comment it out, and Vite will also gracefully behave as if new dependencies were discovered in the middle of an optimization run.

The other fix is to have the block for the noDiscovery condition initialize optimizationResult the same way as the promise that intializing the depsOptimizer.scanProcessing does. This is needed because the initial optimization run normally does not lead to a commit. It seems like the only proper time to call runOptimizer() directly is when the crawl ends.

vite/packages/vite/src/node/optimizer/optimizer.ts

Lines 247 to 251 in 45c6f3b

	// For dev, we run the scanner and the first optimization
	// run on the background, but we wait until crawling has ended
	// to decide if we send this result to the browser or we need to
	// do another optimize step
	optimizationResult = runOptimizeDeps(config, knownDeps)

TL;DR calling runOptimizer() will always lead to committing dependencies, regardless of any in-flight requests. When noDiscovery is set to true it's called twice - once at server start, and again when the first request is transformed.

Reproduction

This issue is proving difficult to extract from the impacted project since it relies on having the initial optimization run taking a substantial amount of time to finish.

Steps to reproduce

This issue is difficult to isolate, but I think it should be reproducible under the following conditions:

1. start a ViteDevServer with optimizeDeps enabled and have both noDiscovery: true and force: true
   a. the project may need a nontrivial number of optimized dependencies since optimizing need to last longer than it takes initDepsOptimizer() to resolve. The project that I've been debugging on transforms ~40 UI components.
2. immediately after the server.listen() completes, call server.transformRequest('/@vite/env') within the same event loop.

System Info

System:
    OS: macOS 12.6.3
    CPU: (10) arm64 Apple M1 Pro
    Memory: 105.56 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.20.0 - ~/.local/share/homebrew/bin/node
    Yarn: 3.5.0 - ~/.local/share/homebrew/bin/yarn
    npm: 8.19.4 - ~/.local/share/homebrew/bin/npm
  Browsers:
    Firefox: 102.12.0
    Safari: 16.3

Used Package Manager

npm

Logs

10:55:49 AM [vite] error while updating dependencies:
Error: ENOENT: no such file or directory, open '/Users/herockk/Workspaces/.../node_modules/.vitest/deps_temp_0722f042/_metadata.json'
    at Object.openSync (node:fs:590:3)
    at Object.writeFileSync (node:fs:2202:35)
    at Object.commit (file:///Users/herockk/Workspaces/.../node_modules/vitest/node_modules/vite/dist/node/chunks/dep-e8f070e8.js:44267:18)
    at commitProcessing (file:///Users/herockk/Workspaces/.../node_modules/vitest/node_modules/vite/dist/node/chunks/dep-e8f070e8.js:43699:40)
    at runOptimizer (file:///Users/herockk/Workspaces/.../node_modules/vitest/node_modules/vite/dist/node/chunks/dep-e8f070e8.js:43737:23)
    at onCrawlEnd (file:///Users/herockk/Workspaces/.../node_modules/vitest/node_modules/vite/dist/node/chunks/dep-e8f070e8.js:43934:17)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[kherock]

@bluwy there's no rush, but would appreciate some feedback on the linked PR soon! There's much more pared down description of the problem there in case this issue contains too much context for a quick review.

[bluwy]

Looks like patak got a hold on it 🙏 Was a bit busy yesterday.