feat: support specPattern, deprecate integrationFolder and componentFolder #19168

lmiller1990 · 2021-12-01T10:18:10Z

Documented here (Cypress team only): https://www.notion.so/cypressdx/testFiles-specPattern-glob-patterns-295d545a0af942ebb5b71eda2a6d422b

This is a gigantic change that touches every inch of the system, and has several breaking changes. Those are:

deprecate testFiles, integrationFolder, component
introduce specPattern, configurable on component and e2e
ignoreTestFiles is now ignoreSpecPattern

Before

{
  integrationFolder: 'cypress/integration',
  testFiles: '**/*'
  ignoreTestFiles: '**.hot'
}

After

{
  e2e: {
    specPattern: 'cypress/integration/**/*',
    ignoreSpecPattern: '**.hot'
  }
}

Scaffolding `cypress/integration` is gone

Our scaffolding logic is very closely tied to cypress/integration existing. Since we no longer scaffold by default on first use, instead using a visual wizard to let the user choose which specs they'd like to scaffold, I deleted all this code and relevant tests.

`util/spec.ts` is gone

Now using the findSpecs method in data-context, which does the same thing with much less indirection, not relying on integration or component folders any more, instead just using the new specPattern property.

Breaking change for `@cypress/code-coverage`

No integrationFolder is a breaking change for code coverage. I patched it locally to get CI passing for now: https://github.com/cypress-io/cypress/pull/19168/files#diff-f7d0849fee5886e2b2efea84707ab2d2b8046975155572be66a486e5a7de1354. We will need to figure out the migration strategy here. Made an issue here.

Default is still `cypress/integration/**/*` instead of `cypress/e2e/**/*.cy.js`

As outlined in notion, we are changing the default directory for e2e from cypress/integration to cypress/e2e. Changing the default means changing all our projects in system-tests/projects (all 89 of them) and all the ones in npm. It's easy but adds a lot of noise to the PR. For this reason, I'll do those in a separate PR.

cypress · 2021-12-02T03:27:48Z

Test summary

18168 • 0 • 202 • 0 • 3

Run details


Project	cypress
Status	Passed
Commit	`ced245e`
Started	Dec 9, 2021 6:10 AM
Ended	Dec 9, 2021 6:22 AM
Duration	11:52 💡
OS	Linux Debian - 10.10
Browser	Multiple

View run in Cypress Dashboard ➡️

Flakiness

	commands/net_stubbing_spec.ts	2
1	network stubbing > waiting and aliasing > can timeout waiting on a single request using "alias.request"
2	network stubbing > waiting and aliasing > can timeout waiting on a single request using "alias.request"
	cypress/proxy-logging-spec.ts	1
1	Proxy Logging > request logging > xhr log has response body/status code

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

…cypress-io/cypress into lmiller1990/update-spec-pattern

lmiller1990 · 2021-12-07T13:33:34Z

packages/config/lib/options.ts

@@ -110,7 +105,9 @@ const resolvedOptions: Array<ResolvedConfigOption> = [
  }, {
    name: 'e2e',
    // e2e runner overrides
-    defaultValue: {},
+    defaultValue: {
+      specPattern: 'cypress/integration/**/*',


This should be cypress/e2e/**/*.cy.js but changing this means changing all our example projects, making this PR ungodly huge, so I will come back for this soon.

Also @tgriesser is doing work around config and this code will change a bunch soon (again) anyway.

lmiller1990 · 2021-12-07T13:34:58Z

packages/graphql/src/schemaTypes/objectTypes/gql-CurrentProject.ts

+          return []
+        }
+
+        const pattern = await ctx.project.specPatternForTestingType(source.projectRoot, ctx.appData.currentTestingType)


should cache these instead of hitting filesystem each time but outside scope of this PR

lmiller1990 · 2021-12-07T13:40:54Z

packages/server/test/integration/http_requests_spec.js

@@ -1137,7 +944,7 @@ describe('Routes', () => {
  //       .expect("Content-Type", /application\/json/)
  //       .expect(JSON.parse(json))

-  context('GET /__cypress/iframes/*', () => {
+  xcontext('GET /__cypress/iframes/*', () => {


I deleted the code this is associated with, but going to leave this in case I missed something.

lmiller1990 · 2021-12-07T13:42:14Z

packages/server/test/unit/modes/run_spec.js

 const { experimental } = require(`${root}../lib/experiments`)

-describe('lib/modes/run', () => {
+// todo(lachlan): put these back in when we've updated run.js
+xdescribe('lib/modes/run', () => {


This spec is basically impossible to get passing without a HUGE amount of extra stubbing, since we now use the data context in run.js... the unit tests here are really not very useful anyway, but I'm going to leave them here and hopefully keep some of them when we finish run mode for the unified runner.

tgriesser · 2021-12-07T14:29:38Z

system-tests/__snapshots__/new_project_spec.js

@@ -39,7 +40,8 @@ exports['e2e new project passes 1'] = `
  (Video)

  -  Started processing:  Compressing to 32 CRF                                                     
-  -  Finished processing: /XXX/XXX/XXX/cypress/videos/app_spec.js.mp4                     (X second)
+  -  Finished processing: /XXX/XXX/XXX/cypress/videos/cypress/integration/app_spec.js     (X second)


I know this probably correct, but it somehow feels off... cypress/videos/cypress/integration vs cypress/videos/integration

cypress/videos + <spec relative to projectRoot>

Previously we stripped of integrationFolder. Yes this feels weird, I agree.

https://cypressio.slack.com/archives/CNM0FKJFJ/p1638886882133300?thread_ts=1638886612.133100&cid=CNM0FKJFJ

tgriesser · 2021-12-07T14:31:59Z

circle.yml

-  # testing scaffolding examples and running them
-  # against example.cypress.io


Do we need to record this as an action item to follow up on / add back in one way or another?

Hm yes probably, we want to make sure all the scaffolded specs still function and work, just without all the complexity of the existing scaffolding setup.

https://cypress-io.atlassian.net/browse/UNIFY-657

lmiller1990 · 2021-12-09T11:40:20Z

gone in favor of #19319

lmiller1990 added 2 commits December 1, 2021 19:54

use new specPattern API

e020e7d

remove projectApi.findSpecs

df3d841

estrada9166 mentioned this pull request Dec 1, 2021

feat: handle testFiles depending of testing type #19075

Closed

5 tasks

lmiller1990 force-pushed the lmiller1990/update-spec-pattern branch from 5ea8a5e to df3d841 Compare December 2, 2021 01:58

lmiller1990 added 3 commits December 2, 2021 11:59

do not require integration folder

acfaf68

support --spec

7c21ce3

support --spec

f380d09

lmiller1990 added 22 commits December 2, 2021 13:29

remove old code

38578c6

remove old code

09f6c6d

nuke old code

f94acf5

no appvetor

692247b

update

9db1091

correct url for ct

7b081a9

work on migrating launchpad

3947f26

update ct spec url

9933ea5

types

8db54d0

types

2ca354b

dead code

97a4a00

remove old endpiont

de7a624

revert back circle.yml

2a801eb

update missing config

983c2c4

delete spec util

b39953a

update config

99eaace

update config

d48b274

config again

218fb5d

update spec pattern

882a06e

updated vue config

1391927

update spec pattern for ui components

8551796

update config for vite dev server

102376e

lmiller1990 added 12 commits December 7, 2021 11:03

Merge branch 'lmiller1990/update-spec-pattern' of https://github.com/…

a620654

…cypress-io/cypress into lmiller1990/update-spec-pattern

merge in origin/10.0-release

052bf82

updated snapshot

283ca1b

change pattern

300ef6f

guard

5e03714

fix smoke test

fb215eb

patch code coverage

30d3455

update percy config

a62098c

fix specs

0455aea

try updating example project

0f95501

update snapshots

d24a522

remove old test

cd6c7be

lmiller1990 changed the title ~~feat: specPattern~~ feat: support specPattern, deprecate integrationFolder and componentFolder Dec 7, 2021

lmiller1990 added 2 commits December 7, 2021 23:21

remove snapshot hack

23465e7

add back appveyor

1d7ed8c

lmiller1990 commented Dec 7, 2021

View reviewed changes

lmiller1990 and others added 3 commits December 7, 2021 23:54

remove old code

9c7b809

update snapshot

db12ca6

Fix tests

84aac5e

jennifer-shehane self-requested a review December 7, 2021 14:23

tgriesser reviewed Dec 7, 2021

View reviewed changes

davidmunechika requested review from BlueWinds and emilyrohrbough December 8, 2021 16:33

merge in origin

ced245e

lmiller1990 closed this Dec 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support specPattern, deprecate integrationFolder and componentFolder #19168

feat: support specPattern, deprecate integrationFolder and componentFolder #19168

lmiller1990 commented Dec 1, 2021 •

edited

cypress bot commented Dec 2, 2021 •

edited

lmiller1990 Dec 7, 2021 •

edited

lmiller1990 Dec 7, 2021

lmiller1990 Dec 7, 2021

lmiller1990 Dec 7, 2021

tgriesser Dec 7, 2021

lmiller1990 Dec 7, 2021

tgriesser Dec 7, 2021

lmiller1990 Dec 7, 2021

lmiller1990 Dec 7, 2021

lmiller1990 Dec 7, 2021

lmiller1990 commented Dec 9, 2021 •

edited by tgriesser

		# testing scaffolding examples and running them
		# against example.cypress.io

feat: support specPattern, deprecate integrationFolder and componentFolder #19168

feat: support specPattern, deprecate integrationFolder and componentFolder #19168

Conversation

lmiller1990 commented Dec 1, 2021 • edited

Before

After

Scaffolding cypress/integration is gone

util/spec.ts is gone

Breaking change for @cypress/code-coverage

Default is still cypress/integration/**/* instead of cypress/e2e/**/*.cy.js

cypress bot commented Dec 2, 2021 • edited

Test summary

Run details

Flakiness

lmiller1990 Dec 7, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lmiller1990 commented Dec 9, 2021 • edited by tgriesser

lmiller1990 commented Dec 1, 2021 •

edited

Scaffolding `cypress/integration` is gone

`util/spec.ts` is gone

Breaking change for `@cypress/code-coverage`

Default is still `cypress/integration/**/*` instead of `cypress/e2e/**/*.cy.js`

cypress bot commented Dec 2, 2021 •

edited

lmiller1990 Dec 7, 2021 •

edited

lmiller1990 commented Dec 9, 2021 •

edited by tgriesser