fix: allow running tests outside Vite project root folder

[IlCallo]

• Reference Component spec file not found by runner (runner uses full path?) - GET /__cypress/iframes/... 404 #22505 (I'm not sure it solves all the problems in that issue)

Additional details

See explanation here: #22505 (comment)

Note that I'm not sure where @fs comes from, probably a Vite plugin?

Steps to test

You'll need Yarn 1, unluckily we're still stuck with that

• clone https://github.com/IlCallo/quasar and checkout upgrade-cypress-ae
• cd ui
• yarn install
• cd dev
• yarn install
• cd ..
• yarn test:component
• notice Cypress can correctly detect tests
• click on any test, I did my tests on QAvatar ones
• notice the request returns 404, due to a wrong relative path composition

How has the user experience changed?

We're now able to reference tests outside Vite project root folder in Quasar monorepo

PR Tasks

• Have tests been added/updated?
• Has a PR for user-facing changes been opened in cypress-documentation?
• Have API changes been updated in the type definitions?

[CLAassistant]

All committers have signed the CLA.

[cypress]

41 flaky tests on run #44062 ↗︎

0

26802

1271

0

41

Details:

fix: allow running tests outside Vite project root folder
Project: cypress	Commit: 9db5251a0e
Status: Passed	Duration: 20:01 💡
Started: Feb 14, 2023 8:49 AM	Ended: Feb 14, 2023 9:09 AM

  create-from-component.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
... > runs generated spec		Screenshot

  specs_list_latest_runs.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
App/Cloud Integration - Latest runs and Average duration > when no runs are recorded > shows placeholders for all visible specs		Screenshot

  cypress-in-cypress-component.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
Cypress In Cypress CT > default config > redirects to the specs list with error if a spec is not found		Screenshot

  cypress-origin-communicator.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
Cypress In Cypress Origin Communicator > cy.origin passivity with app interactions > passes upon test reload mid test execution		Screenshot

  commands/net_stubbing.cy.ts • 1 flaky test • 5x-driver-firefox

View Output Video

Test
network stubbing > intercepting request > can delay and throttle a StaticResponse

The first 5 flaky specs are shown, see all 21 specs in Cypress Cloud.

_{This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.}

[ZachJW34]

@IlCallo Could you run SNAPSHOT_UPDATE=1 yarn workspace @tooling/system-tests test vite_dev_server_fresh_spec (after running yarn and making sure everything is built)? It should update our snapshot file and then you can push those up in a separate commit to get the tests to pass.

Also, we have an explicit fs.allow list

cypress/npm/vite-dev-server/src/resolveConfig.ts

Line 111 in 9db5251

allow: [

so I'm surprised your app is working without modifying this. We might be able to add fs.strict: false here to just allow all. Any thoughts?

Would also be good to get a test demonstrating this new behavior which I can help you with.

[lmiller1990]

Copy pasting the reproduction @IlCallo supplied in Discord for some context

Notes about the repro:

- you'll need Yarn 1, unluckily we're still stuck with that
- clone https://github.com/IlCallo/quasar and checkout upgrade-cypress-ae
- cd ui
- yarn install
- cd dev
- yarn install
- cd ..
- yarn test:component
- notice Cypress can correctly detect tests
- click on any test, I did my tests on QAvatar ones
- notice the request returns 404, probably due to a wrong path 

Not sure if this fixes all issues in #22505, but it looks like it at least fixes Quasar, who ships us as part of their first class tooling, so I'd definitely like to see this one reviewed and merged (unless it is introduces a regression).

[lmiller1990]

Looks like @fs is a Vite thing - I couldn't find much documentation outside of https://vitejs.dev/config/server-options.html#server-fs-allow.

I'll update the snapshots - I don't know if this breaks anything, but I always thought the src/src duplication was kind of weird.

[lmiller1990]

I took care of updating the snapshots ecdabcd

@ZachJW34 do you have any thoughts on the blast radius of this change? I suspect this is an implementation that shouldn't impact anyone (other than people experiencing a related bug, for whom it'll fix).

[IlCallo]

so I'm surprised your app is working without modifying this

I'm surprised too, but I'm not a Vite expert, so there's probably some underlying business rule I'm not aware of.
Could it be due to the fact that we provide our own vite dev server config?

[ZachJW34]

@lmiller1990 I think the blast radius is small. Stack traces are working properly and it fixes the bug (checked out the quasar repro to verify). It was cool to see that the specs list works as expected for file outside the projectRoot (with the ../)!

I'm good with this change!

[lmiller1990]

Thanks @IlCallo, this will go in the next release, that'll be in ~2 weeks. I'll merge it up by the end of this week (need to do a few housekeeping things, like a changelog entry, etc).

[IlCallo]

@ZachJW34 @lmiller1990
I dug a bit further and found out why it works for Quasar repo tests: we tamper with the viteConfig of our project adding .. into server.fs.allow

https://github.com/quasarframework/quasar/blob/8f189d3f6b54d083cb40da8bf809d38dd120faa5/ui/dev/quasar.config.js#L80-L82

The change in this PR is still relevant, but it may be a good idea to specify somewhere on the docs that customizing the server.fs.allow property is needed to access files outside what Cypress consider the project root folder

Unless there are security concerns, removing the server.fs.allow altogether could be an option too.
I guess it depends on why it has been added in the first place

[lmiller1990]

I suspect we should not tamper with server.fs.allow, since it's related to opt in file system access (I think we a third party library should not mess with this) but we can definitely add a note to the docs around this.

[IlCallo]

suspect we should not tamper with server.fs.allow

Playing the guess game, I'd say you added that option because since Vite 2.7 that option is enabled by default and only allows to access files in the current workspace(s), but you have some files you need which are outside the project root folder (as we have), especially the Cypress binary stored in the global cache

Another related problem which recently came up is that you aren't re-adding the workspaces paths to the allow after overriding the option, which cause problems when using Cypress component testing to execute all tests into a monorepo with multiple packages: #26036

[mike-plummer]

@lmiller1990 Potentially related issue & PR opened to work around the server.fs.allow prop in monorepos, in case you're curious