chore: prep npm packages for use with Cypress v10 #22205

ZachJW34 · 2022-06-08T21:03:40Z

Closes Release: Ensure we are publishing the correct versions of CT related npm packages for upcoming 10.0 release #21944

User facing changelog

Publish Cypress v10 compatible npm packages

Additional details

Removes "private": true from the npm packages that we had disabled and fixes a few bugs in our implementation that would have caused issues. Notable changes:

Replacing the require path for @cypress/webpack-batteries-included-preprocessor in webpack-dev-server from __dirname to cypressBinaryRoot. Relying on __dirname only works when sourcing from within the binary (or sourcing from the packages installed node_modules).
Revert find-up to 5.0.0 as >6.0.0 is ESM only. We were relying on local-pkg to import it which uses require. Removed both packages.
Restoed webpack-dev-server changelog that got removed in a previous merge
Fixed types for when using the dev-server function signature

The scope for the original issue became very bloated so I'm not ticking all the boxes from that issue (such as linting non-used deps or releasing as RC). Not sure why we came to the decision to publish them as RC but I thoroughly tested all the packages locally and I'm confident we can go forward with a normal breaking change.

I ran the npm-release.js script locally and the new versions are as expected:

node ./scripts/npm-release.js 
Found the following public packages: create-cypress-tests, @cypress/schematic, @cypress/eslint-plugin-dev, @cypress/mount-utils, @cypress/react, @cypress/vite-dev-server, @cypress/vue, @cypress/vue2, @cypress/webpack-batteries-included-preprocessor, @cypress/webpack-dev-server, @cypress/webpack-preprocessor

Finding package versions...

Cypress binary: 10.0.3

create-cypress-tests
Current version: 1.3.0
Next version: 2.0.0

@cypress/schematic
Current version: 2.0.0
Next version: N/A

@cypress/eslint-plugin-dev
Current version: 5.3.0
Next version: N/A

@cypress/mount-utils
Current version: 1.0.2
Next version: 2.0.0

@cypress/react
Current version: 5.12.5
Next version: 6.0.0

@cypress/vite-dev-server
Current version: 2.2.3
Next version: 3.0.0

@cypress/vue
Current version: 3.1.2
Next version: 4.0.0

@cypress/vue2
Current version: N/A
Next version: 1.0.0

@cypress/webpack-batteries-included-preprocessor
Current version: 2.2.3
Next version: N/A

@cypress/webpack-dev-server
Current version: 1.8.4
Next version: 2.0.0

@cypress/webpack-preprocessor
Current version: 5.12.0
Next version: N/A

Found a new release for the following packages: create-cypress-tests, @cypress/mount-utils, @cypress/react, @cypress/vite-dev-server, @cypress/vue, @cypress/vue2, @cypress/webpack-dev-server

We should merge with care. I included two commits, one with all the changes to the files in npm/ with the BREAKING CHANGE commit message and another with the rest of the changes. I didn't want to include a BREAKING CHANGE commit message that touched files outside of npm. When merging, I'd like to not squash.

Need to follow up with DX about documentation. There is a section about custom-dev-servers, gotta check if this needs to be expanded.

Steps to test

I tested the packages by running yarn pack and installing the generated .tgz. I added a few system tests for the dev-servers to verify the use of the function signature.

How has the user experience changed?

na

PR Tasks

Have tests been added/updated?
Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
Has a PR for user-facing changes been opened in cypress-documentation?
Have API changes been updated in the type definitions?

cypress-bot · 2022-06-08T21:03:42Z

Thanks for taking the time to open a PR!

Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.
Become familiar with the Code Review Checklist for guidelines on coding standards and what needs to be done before a PR can be merged.

BlueWinds · 2022-06-08T21:14:16Z

npm/webpack-dev-server/src/makeWebpackConfig.ts

    configFile = await findUp(configFiles, { cwd: projectRoot } as { cwd: string })

    if (configFile) {
      debug('found webpack config %s', configFile)
-      const sourcedConfig = await importModule(configFile)
+      const sourcedConfig = configFile.endsWith('.mjs') ? await import(configFile) : require(configFile)


This make me nervous. Have you tried this with a project with type: module in package.json? In my experience extension is very unreliable when determining how to import things.

Other than this one change, LGTM.

Yeah I was reverting to what was there before, but I confirmed it doesn't work. Turns out the TS compilation will convert import to a require which is not what we want (see here). There are some workarounds, I'll check both webpack and vite since we do similar things for loading the config

Going over this PR now -- we can sync on this if needed, I've been dealing with module stuff lately. Not saying I have a perfect solution, but we can explore some things.

If we change the tsconfig to "module": "esnext" it will not generate require, but maintain the original import.

Actually, thinking about it, we have the exact same problem in packages/server, where we need to grab their cypress.config, which I've been banging my head on the wall over for the last three days. User can have a cypress.config can be:

"type": "module" with js

"type": "CommonJS" (the default) with .mjs

.ts with "module": "CommonJS" in tsconfig.json

.ts with "module": "esnext" in tsconfig.json

It's also possible to have .cjs and, believe it or not, .mjs and .mts - it's pretty impractical, right now, but we need to cover as many as possible and fail as gracefully as possible.

I think we probably want to have one single, uniform way to handle "all the things" if possible. The only difference, I guess, would be that the binary ships ts-node, so we know we can use that, but npm/webpack-dev-server cannot rely on ts-node been installed.

The obvious edge case here:

configFile = "cypress.config.ts" const sourcedConfig = configFile.endsWith('.mjs') ? await import(configFile) : require(configFile)

Now we do require("cypress.config.ts"), which is obviously invalid unless you are using ts-node, which they might (probably) won't be ... require is for CJS only, so it won't work with 1) ts and even if it did, most TS files are using import syntax.

packages/server handles this with ts-node/esm, so it'd be fine there - but for a user land module like this, they'll have a bad day. We can definitely just document what is/isn't supported, or make a recommendation to use ts-node etc. I think we will need to think about this a bit more - what are you thoughts?

cypress · 2022-06-08T21:19:33Z

Test summary

4938 • 0 • 58 • 0 • 2

Run details


Project	cypress
Status	Passed
Commit	`868081a`
Started	Jun 13, 2022 5:38 PM
Ended	Jun 13, 2022 5:55 PM
Duration	16:16 💡
OS	Linux Debian - 10.11
Browser	Multiple

View run in Cypress Dashboard ➡️

Flakiness

	cypress/e2e/next.cy.ts		2
1	Working with next-11-webpack-4 > should live-reload on src changes
2	Working with next-12.1.6 > should detect new spec

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

lmiller1990 · 2022-06-09T02:24:04Z

npm/vite-dev-server/cypress/e2e/vite-dev-server.cy.ts

@@ -48,4 +47,15 @@ describe('Config options', () => {
    cy.contains(specWithWhitespace).click()
    cy.get('.passed > .num').should('contain', 1)
  })
+
+  it('supports @cypress/vite-dev-server', () => {


What does this test actually prove works?

The system test is hooked up the use import { devServer } from '@cypress/vite-dev-server via the file: package alias, so it's a test to prove that importing this as a standalone package and using the function signature works.

lmiller1990

Left some general comments - I'm wondering if we don't revert the module loading changes you made and do them separately (eg, this ticket was just about "release the things" not necessarily fixing bugs?

lmiller1990 · 2022-06-09T02:27:00Z

npm/vite-dev-server/package.json

@@ -17,8 +16,7 @@
  },
  "dependencies": {
    "debug": "4.3.3",
-    "find-up": "6.3.0",
-    "local-pkg": "0.4.1",
+    "find-up": "5.0.0",


Not a blocker, but I really like the idea of moving (slowly) towards ES Modules.

I wonder what the logistics of:

Write vite-dev-server using "type": "module" (with tsconfig.json using "module": "esnext")

Now the package is 100% authored using ES modules.

Use rollup to generate a CJS and ESM bundle.

Haven't looked much into ESM source -> ESM/CJS combined output. I know a bunch of modules like find-up just ripped the bandaid off an took a hard "ESM only" stance. While we cannot reasonably do that, the idea of at least authoring in ES Modules is appealing, and pushing the CommonJS concern to a build-step-only thing.

One thing to consider here is that these packages are always going to be consumed through our loader process since they are imported in the config (which is processed in the child process we spin up). Not quite sure of all the implications of this, but if we always transpile the config to commonjs then it might factor into how we decide to bundle these packages.

I'm seeing the need for a clearly defined tech-brief just talking through all of this!

Ah yes, we'd need to change our development workflow to transpire this on the fly if we wanted to write pure ESM (probably not a big deal, but more work than really makes sense now).

That said, unless we eventually do commit to it, we will likely be stuck on ESM forever (like most of the ecosystem). We finally got the module system we all dreamed off, but it came 10 years too late :(

lmiller1990 · 2022-06-09T02:29:59Z

npm/webpack-dev-server/src/makeWebpackConfig.ts

    configFile = await findUp(configFiles, { cwd: projectRoot } as { cwd: string })

    if (configFile) {
      debug('found webpack config %s', configFile)
-      const sourcedConfig = await importModule(configFile)
+      const sourcedConfig = configFile.endsWith('.mjs') ? await import(configFile) : require(configFile)


The obvious edge case here:

configFile = "cypress.config.ts" const sourcedConfig = configFile.endsWith('.mjs') ? await import(configFile) : require(configFile)

Now we do require("cypress.config.ts"), which is obviously invalid unless you are using ts-node, which they might (probably) won't be ... require is for CJS only, so it won't work with 1) ts and even if it did, most TS files are using import syntax.

packages/server handles this with ts-node/esm, so it'd be fine there - but for a user land module like this, they'll have a bad day. We can definitely just document what is/isn't supported, or make a recommendation to use ts-node etc. I think we will need to think about this a bit more - what are you thoughts?

ZachJW34 · 2022-06-10T21:00:48Z

@lmiller1990 @BlueWinds I decided to revert the changes made to how the webpack config was being loaded. I wrote a decent amount of test cases to cover all the variations of js + cjs + ts + esm and they were pros and cons for each approach. The way it is wired up now is actually the best for our use case of providing good support for the most common cases (js + ts + cjs).

I've learned that there is a workaround for esm and ts edge-cases which is to import the webpack config into the cypress config i.e. not rely on @cypress/webpack-dev-server. For example, if the cypress config is loaded as type=module and the webpack config is imported there it is processed with the right context (same with typescript).

We have some work to do for handling esm, and I'm going to push up my test cases and link them to a new issue where we can properly work through adding first class support for esm. I feel better having this issue just be related to publishing the packages rather than solve the nightmare that is support for cjs + esm.

lmiller1990

Cool, good call on not doing too deep in the module stuff. #21939 seems to be working pretty well, so we can expand on that approach moving forward. Let's work together on a brief to figure this out once and for all.

Windows CI is fixed in develop, you might need to wait for develop to merge into master before this is able to be merged.

ZachJW34 · 2022-06-13T16:40:25Z

Going to merge today, there is a CI issue that I'm merging to develop and will get merged into master soon: #22289

BREAKING CHANGE: new version of packages for Cypress v10

ZachJW34 · 2022-06-13T18:04:59Z

Percy-finalize having an issue, all test are passing so going to merge.

ZachJW34 requested review from tgriesser, elylucas and lmiller1990 June 8, 2022 21:03

ZachJW34 requested review from a team as code owners June 8, 2022 21:03

ZachJW34 requested review from jennifer-shehane and removed request for a team June 8, 2022 21:03

ZachJW34 requested a review from BlueWinds June 8, 2022 21:04

BlueWinds suggested changes Jun 8, 2022

View reviewed changes

lmiller1990 reviewed Jun 9, 2022

View reviewed changes

ZachJW34 force-pushed the zachw/ensure-ct-10.0-packages branch 2 times, most recently from 274b700 to 722e838 Compare June 10, 2022 20:42

ZachJW34 requested a review from BlueWinds June 10, 2022 21:03

lmiller1990 approved these changes Jun 13, 2022

View reviewed changes

ZachJW34 force-pushed the zachw/ensure-ct-10.0-packages branch from 722e838 to 77cfadd Compare June 13, 2022 14:45

BlueWinds approved these changes Jun 13, 2022

View reviewed changes

ZachJW34 mentioned this pull request Jun 13, 2022

fix: remove unused system-test project #22289

Merged

ZachJW34 added 2 commits June 13, 2022 12:06

chore: prep npm packages for use with Cypress v10

b924d08

BREAKING CHANGE: new version of packages for Cypress v10

test: update dev-server types and add tests

868081a

ZachJW34 force-pushed the zachw/ensure-ct-10.0-packages branch from 77cfadd to 868081a Compare June 13, 2022 17:07

ZachJW34 merged commit 555f7cd into master Jun 13, 2022

ZachJW34 deleted the zachw/ensure-ct-10.0-packages branch June 13, 2022 18:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: prep npm packages for use with Cypress v10 #22205

chore: prep npm packages for use with Cypress v10 #22205

ZachJW34 commented Jun 8, 2022

cypress-bot bot commented Jun 8, 2022

BlueWinds Jun 8, 2022 •

edited

ZachJW34 Jun 8, 2022 •

edited

lmiller1990 Jun 9, 2022 •

edited

lmiller1990 Jun 9, 2022

cypress bot commented Jun 8, 2022 •

edited

lmiller1990 Jun 9, 2022

ZachJW34 Jun 9, 2022

lmiller1990 left a comment

lmiller1990 Jun 9, 2022

ZachJW34 Jun 9, 2022

lmiller1990 Jun 10, 2022

lmiller1990 Jun 9, 2022

ZachJW34 commented Jun 10, 2022

lmiller1990 left a comment •

edited

ZachJW34 commented Jun 13, 2022

ZachJW34 commented Jun 13, 2022

chore: prep npm packages for use with Cypress v10 #22205

chore: prep npm packages for use with Cypress v10 #22205

Conversation

ZachJW34 commented Jun 8, 2022

User facing changelog

Additional details

Steps to test

How has the user experience changed?

PR Tasks

cypress-bot bot commented Jun 8, 2022

BlueWinds Jun 8, 2022 • edited

Choose a reason for hiding this comment

ZachJW34 Jun 8, 2022 • edited

Choose a reason for hiding this comment

lmiller1990 Jun 9, 2022 • edited

Choose a reason for hiding this comment

lmiller1990 Jun 9, 2022

Choose a reason for hiding this comment

cypress bot commented Jun 8, 2022 • edited

Test summary

Run details

Flakiness

lmiller1990 Jun 9, 2022

Choose a reason for hiding this comment

ZachJW34 Jun 9, 2022

Choose a reason for hiding this comment

lmiller1990 left a comment

Choose a reason for hiding this comment

lmiller1990 Jun 9, 2022

Choose a reason for hiding this comment

ZachJW34 Jun 9, 2022

Choose a reason for hiding this comment

lmiller1990 Jun 10, 2022

Choose a reason for hiding this comment

lmiller1990 Jun 9, 2022

Choose a reason for hiding this comment

ZachJW34 commented Jun 10, 2022

lmiller1990 left a comment • edited

Choose a reason for hiding this comment

ZachJW34 commented Jun 13, 2022

ZachJW34 commented Jun 13, 2022

BlueWinds Jun 8, 2022 •

edited

ZachJW34 Jun 8, 2022 •

edited

lmiller1990 Jun 9, 2022 •

edited

cypress bot commented Jun 8, 2022 •

edited

lmiller1990 left a comment •

edited