experimental testmode unstable

[brunolca]

Link to the code that reproduces this issue

https://github.com/brunolca/reproduction-next-experimental-testmode

To Reproduce

1. pnpm i
2. pnpm test

When running the test for the first time, you might not have this issue. To reproduce it, try to uncomment/comment the second fetch in app/page.tsx ...

The reproduction-next-experimental-testmode repo was created with the base reproduction example and following the steps as described here.

Current vs. Expected behavior

When running the playwright test i get the following error :

[WebServer]  ⨯ Error: No test info for GET https://jsonplaceholder.typicode.com/posts/1
    at async Home (./app/page.tsx:9:25)
[WebServer] digest: "1849632151"
[WebServer]  ⨯ Error: No test info for GET https://jsonplaceholder.typicode.com/posts/1
    at async Home (./app/page.tsx:9:25)
[WebServer] digest: "1849632151"
[WebServer]  ⨯ Error: No test info for GET https://jsonplaceholder.typicode.com/posts/1
    at async Home (./app/page.tsx:9:25)
[WebServer] digest: "1849632151"

When running the test for the first time, you might not have this issue. To reproduce it, try to uncomment/comment the second fetch in app/page.tsx ...

I expected the fetch request in app/page.tsx to be mocked and the test to successfully run. Again, be aware that the test can work as expected when running it but it is inconsistent.

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020
  Available memory (MB): 16384
  Available CPU cores: 10
Binaries:
  Node: 21.6.2
  npm: 10.2.4
  Yarn: 1.22.21
  pnpm: 8.15.4
Relevant Packages:
  next: 14.2.0-canary.13 // Latest available version is detected (14.2.0-canary.13).
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.1.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

App Router

Which stage(s) are affected? (Select all that apply)

next dev (local)

Additional context

@dvoytenko I think you are the one in charge of this. I hope you can help me on this. Thank you

[brunolca]

it appears to be working constantly when building the app before testing even if the the playwright config is running the server in dev mode.

Even with this workaround, i can't make it work when the fetch request happens in the middleware of the app.

[dvoytenko]

@brunolca this looks like the automatic fetch caching issue. Could you modify your fetch to look like this?

await fetch("https://jsonplaceholder.typicode.com/posts/1", {
  cache: "no-store",
})

Let me know if that works for you.

If it does, we could further do one or both of the following:

1. I can document how to disable cache for testing.
2. (Maybe even better) I could automatically disable cache for the --experimental-test-proxy.

[dvoytenko]

#63265 should fix this issue

[github-actions]

This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.