Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Improve error handling around calls to this.next in middleware #25702

Merged
merged 17 commits into from
Feb 8, 2023

Conversation

mike-plummer
Copy link
Contributor

@mike-plummer mike-plummer commented Feb 3, 2023

Additional details

Issue was reported in v10.3.0 - a CT test structured in a very particular way could get a "this.next() function getting called twice within a given middleware invocation" error message. This was due to bad error-handling logic within Cypress and disguised an actual error in the CT server instance. This particular error condition is fixed in this PR specifically for component tests.

Other users reported similar error messages in v10.x and v11.2.0 in e2e tests but were unable to provide a reproduction or debug logs. I have been unable to reproduce the issue in an e2e test in v12.5.1 - there's' been changes to this area of Cypress recently so it's possible this was "accidentally fixed". In case it was not I'm adding more error logging - this should help identify the root error and give us additional info if it happens to be a Cypress bug (or allow the user to fix issues in their own config/logic).

Note: This additional error logging is not strictly necessary since the correct error is being exposed if you print a network request failure to devtools console from the command log, enable DEBUG logging, or inspect from a cy.intercept, but those troubleshooting steps may not be intuitive for this sort of error based on how the original issue unfolded. I'm open to opinions on how to handle this if the logging feels too heavy-handed.

Steps to test

Clone reproduction case, follow instructions in README

This example shows current behavior, new behavior with fix for internal error, and new behavior with improved error logging

How has the user experience changed?

  1. Fixed potential internal error in middleware for CT tests
  2. Added logging to terminal for errors propagating out of middleware so they are more evident - user could previously access if they inspected the failed network call from command log or printed from a cy.intercept but this should make them more evident
  3. Added logging to "double this.next invocation" condition to help identify if a previously-handled error may have contributed - this should help diagnose bad middleware customizations or intercept logic

PR Tasks

  • Have tests been added/updated?
  • Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
  • [na] Has a PR for user-facing changes been opened in cypress-documentation?
  • [na] Have API changes been updated in the type definitions?

@cypress
Copy link

cypress bot commented Feb 6, 2023

39 flaky tests on run #43899 ↗︎

0 26786 1271 0 Flakiness 39

Details:

Fix unit tests
Project: cypress Commit: fbd5d51590
Status: Passed Duration: 18:18 💡
Started: Feb 7, 2023 9:10 PM Ended: Feb 7, 2023 9:29 PM
Flakiness  e2e/origin/cookie_behavior.cy.ts • 4 flaky tests • 5x-driver-electron

View Output Video

Test
... > same site / cross origin > XMLHttpRequest > sets cookie on same-site request if withCredentials is true, and attaches to same-site request if withCredentials is true
... > same site / cross origin > fetch > sets same-site cookies if "include" credentials option is specified from request, but does not attach same-site cookies to request by default (same-origin)
... > same site / cross origin > XMLHttpRequest > sets cookie on same-site request if withCredentials is true, and attaches to same-site request if withCredentials is true
... > same site / cross origin > fetch > sets same-site cookies if "include" credentials option is specified from request, but does not attach same-site cookies to request by default (same-origin)
Flakiness  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry
Flakiness  create-from-component.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
... > runs generated spec Screenshot
Flakiness  specs_list_latest_runs.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
App/Cloud Integration - Latest runs and Average duration > when no runs are recorded > shows placeholders for all visible specs Screenshot
Flakiness  cypress-in-cypress-component.cy.ts • 1 flaky test • app-e2e

View Output Video

Test
Cypress In Cypress CT > default config > redirects to the specs list with error if a spec is not found Screenshot

The first 5 flaky specs are shown, see all 17 specs in Cypress Cloud.

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@mike-plummer mike-plummer marked this pull request as ready for review February 6, 2023 17:44
@mike-plummer mike-plummer requested review from flotwig, ZachJW34 and a team February 6, 2023 18:21
Copy link
Contributor

@lmiller1990 lmiller1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work to track this down - looks like the original user who reported it did a good job debugging, too.

I did not pull the try the reproduction yet, but the code looks good - left a few minor but non-blocking comments.

packages/proxy/lib/http/index.ts Show resolved Hide resolved
packages/server/lib/server-base.ts Outdated Show resolved Hide resolved
packages/server/lib/server-base.ts Show resolved Hide resolved
packages/proxy/lib/http/index.ts Outdated Show resolved Hide resolved
packages/proxy/lib/http/index.ts Outdated Show resolved Hide resolved
packages/proxy/lib/http/index.ts Show resolved Hide resolved
* Append root error as `cause` on `this.next` error
* Improve error messages
* Remove unneeded type export

if (ctx.error) {
error.message = error.message += '\nThis middleware invocation previously encountered an error which may be related, see `error.cause`'
error['cause'] = ctx.error
Copy link
Contributor

@flotwig flotwig Feb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the type error this works around could be due to us using an outdated @types/node... but that may be more than a 1-line change to update.

It doesn't look like logError will log cause, so I'm not sure this is going to get logged by line 218:

export const logError = function (err: CypressError | ErrorLike, color: AllowedChalkColors = 'red') {
console.log(chalk[color](err.message))
if (err.details) {
console.log(chalk.magenta(`\n${err.details}`))
}
// bail if this error came from known
// list of Cypress errors
if (isCypressErr(err)) {
return
}
console.log(chalk[color](err.stack ?? ''))
return err
}

Copy link
Contributor Author

@mike-plummer mike-plummer Feb 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated logError to recurse down cause (if present) and log to a fixed depth in case there's a loop - rare, but I've been bitten by that in the past

Took a look at updating the @types/node bindings - it would end up touching a 3-4 other packages and my yarn was having serious indigestion trying to bump the version so I may leave as-is with the type workaround just to keep this PR targeted

@mike-plummer mike-plummer merged commit efc1958 into develop Feb 8, 2023
@mike-plummer mike-plummer deleted the mikep/22825-next-middleware branch February 8, 2023 16:17
tgriesser added a commit that referenced this pull request Feb 8, 2023
* develop: (28 commits)
  chore: update changelog validation example (#25742)
  fix: Improve error handling around calls to `this.next` in middleware (#25702)
  chore: debug page tooltip distance and artifact border (#25727)
  fix: update newProject ref when switching between organizations in SelectCloudProjectModal (#25730)
  misc: Add max widths to debug page message states (#25725)
  chore: export types (#25714)
  chore: release @cypress/webpack-preprocessor-v5.16.3
  chore: release @cypress/vue-v5.0.4
  chore: release @cypress/grep-v3.1.4
  chore: Fix flaky test (#25726)
  dependency(deps): update dependency debug to ^4.3.4 🌟 (#25699)
  feat: openInIDE for failed debug spec (#25691)
  test: fix flaky CT test by relying on query (#25706)
  test: fix flaky migration test (#25672)
  misc: style change for responsiveness (#25687)
  misc: set min widths for icons (#25684)
  chore(deps): update dependency markdown-it to v11.0.1 🌟 (#25698)
  chore: Fix flaky origin .wait() test (#25693)
  chore: unskip tests (#25676)
  chore: release @cypress/webpack-preprocessor-v5.16.2
  ...
mjhenkes added a commit that referenced this pull request Feb 21, 2023
* fix: update newProject ref when switching between organizations in SelectCloudProjectModal (#25730)

* chore: debug page tooltip distance and artifact border (#25727)

* misc: debug page tooltip distance and artifact border

* add changelog entry

* fix CT test

* fix: Improve error handling around calls to `this.next` in middleware (#25702)

* chore: update changelog validation example (#25742)

* misc: improve debug loading text wrap responsiveness (#25703)

* misc: Increase max failures in IATR badge to 99 (#25737)

* chore: exclude collaborator issues/PRs from triage project (#25769)

* feat: add --auto-cancel-after-failures flag (#25237)

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: Matt Schile <mschile@cypress.io>
Co-authored-by: Ryan Pei <ryanppei@gmail.com>
Co-authored-by: Emily Rohrbough <emilyrohrbough@yahoo.com>

* chore: Update v8 snapshot cache (#25592)

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* Update update_v8_snapshot_cache.yml

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

---------

Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: cypress-bot[bot] <47117332+cypress-bot[bot]@users.noreply.github.com>

* fix: implement new graphql fields for spec counts (#25757)

Co-authored-by: Stokes Player <stokes@cypress.io>
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>

* feat: Bundle cy.origin() dependencies at runtime (#25626)

Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>

* chore: remove zenhub from release process (#25701)

Co-authored-by: Matt Schile <mschile@cypress.io>

* feat: add Cypress.Commands.overwriteQuery (#25674)

* feat: add Cypress.Commands.overwriteQuery

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: Zach Bloomquist <git@chary.us>

* fix: spawn child process with process.env in macOS arm64 (#25753)

Co-authored-by: Matt Schile <mschile@cypress.io>
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: Zach Bloomquist <github@chary.us>

* chore: lint system tests in CI (#25673)

* fix: Suppress filesystem errors during glob search (#25774)

* chore: issue with ts-loader missing in binary and problematic esbuild norewrite construct (#25797)

* chore: update changelog linting (#25809)

* docs(guides): add more detail to code-signing (#25794)

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>

* chore: update workflows.yml to include the v8 snapshot update branch (#25784)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>

* chore: internal request preflight (#25772)

---------

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: Matt Henkes <mjhenkes@gmail.com>
Co-authored-by: Zach Bloomquist <git@chary.us>

* chore: bump for 12.6.0 release (#25812)

* chore: release @cypress/webpack-batteries-included-preprocessor-v2.4.0

[skip ci]

* chore: release @cypress/webpack-preprocessor-v5.17.0

[skip ci]

* test: skip flaky GitDataSource test (#25825)

* chore: making our add-to-triage-board workflow reusable within the Cypress-io org (#25820)

* chore: Making our add to triage workflow callable from other projects inside the Cypress-io org in Github

* chore: updated cypress-example-kitchensink version (#25828)

* fix: duplicate and expired cookies (#25761)

* chore: add regression tests for duplicate cookies and bad expiry times

* avoid prepending domain with dot for cookies that are set with the server side jar. This is to avoid the cookie being duplicated if it is set or overridden in a different context (request that can actually set the cookie or via document.domain)

* feat: use cookie.toString() in the cookie patch to more accurately set cookies on the document, which should include other properties besides key=value

* fix: add logic to handle expired cookies in the document.cookie patch, as well as in CDP

* chore: build binary for cookie fixes for users to test

* chore: change name of fixture to something more accurate

* chore: comment why we are using the toughCookie toString method in the patch

* [run ci]

* chore: add changelog entry

* [run ci]

* fix: revert back to key=value when getting document.cookie as those are the only values are displayed (oversight on my end)

* [run ci]

* chore: make compatible with cypress.require

* fix: add tests for hostOnly/non hostOnly cookies to make sure property gets sent up to automation client correctly. No longer need custom cookie prop to determine destination

* [run ci]

* fix: stale unit test

* chore: adjust comments

* [run ci]

* fix: bad domain logic

* [run ci]

* chore: remove irrelevant comment

* [run ci]

* fix: adjust cookie login text to spec hostOnly cookie within the cookie patch. This should yield the same behavior as we are bound to same origin within the spec bridge

* [run ci]

* [run ci]

* fix: allow for cookies on request of same key to take precedence over cookies in the jar, regardless of how many hierachy cookies exist in the jar

* chore: fix cookie misc tests for cy.origin (dont run cy.origin)

* [run ci]

* chore: skip misc cookie tests in webkit as headless behavior doesn't clear cookies between tests correctly

* Revert "fix: allow for cookies on request of same key to take precedence over cookies in the jar, regardless of how many hierachy cookies exist in the jar"

This reverts commit 17de188.

* [run ci]

* chore: split changelog entry into two parts

* chore: update logic to remove else statement and add comments

* [run ci]

* chore: readd windows snapshot branch in workflows

* [run ci]

* chore: fix workflows from bad merge

* [run ci]

* Revert "chore: split changelog entry into two parts"

This reverts commit 4352ef5.

* [run ci]

* fix: Fix type definitions for cy.reload() (#25779)

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>

* misc: Debug header updates (#25823)

* fix: allow running tests outside Vite project root folder (#25801)

* fix: allow running tests outside Vite project root folder

* update snapshots

* add changelog entry

---------

Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>

* fix: mount component in [data-cy-root] (#25807)

* fix(angular): mount component in [data-cy-root]

* fix e2e test

* add changelog entry

* changelog [skip ci]

* changelog

---------

Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>

* chore: updating add to triage baord github action to use org secret (#25868)

* chore: updating add to triage board github action to use org secret

* chore: release @cypress/angular-v2.0.2

[skip ci]

* chore: release @cypress/vite-dev-server-v5.0.3

[skip ci]

* chore: Update v8 snapshot cache (#25822)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>

* feat: support host only cookies (#25853)

* feat: allow setCookie API to take a hostOnly option

* chore: add jsdoc/typescript description to render to users

* chore: add changelog entry

* [run ci]

* chore: fix types

* chore: fix cookie login tests

* chore: update e2e cookie system tests

* [run ci]

* chore: fix cookie command tests. localhost cookies are calculated as hostOnly, which is consistent with how cypress works today

* chore: fix system tests for cookies.

* [run ci]

* chore: fix system tests

* chore: skip hostOnly assertions in webkit (for now)

* [run ci]

* chore: add property definitions to setCookieOptions

* [run ci]

* chore: add comments to hostOnly prop in firefox when setting a cookie

* fix(webpack-dev-server): touch component-index during onSpecsChange to avoid writing to app file (#25861)

* testing: try disabling uTimesSync and see what happens

* build binaries [run ci]

* fix: touch component index file instead of browser.js

* build binaries [run ci]

* update test

* fix test

* add test for custom HTML file in config

* use existing component index in webpack-dev-server unit tests

---------

Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>

* chore: release @cypress/webpack-dev-server-v3.2.4

[skip ci]

* chore: improve types for server automation cookie client (#25836)

* chore: improve types for automation cookies

* [run ci]

* fix: the cookie_behavior tests by syncing cookies immediately if … (#25855)

* fix: fix the cookie_behavior tests by syncing cookies immediately if the application is already stable

* chore: add changelog entry

* [run ci]

* chore: address comments from code review

* feat: Public API for CT Framework Definitions (#25780)

* chore: rework component onboarding in launchpad (#25713)

* chore: refactoring and types

* rework source of frameworks

* revert rename

* fix tests

* fix more tests

* types

* update code

* use same public API internally

* rename interfaces

* rename

* work on dev server api

* fix types

* fix test

* attempt to support getDevServerConfig

* tests

* add function to define framework [skip ci]

* rework a lot of types

* fix test

* update tests and types

* refactor

* revert changes

* lint

* fix test

* revert

* remove

* add "community" label [skip ci]

* refactor

* types

* lint

* fix bug

* update function name

* address feedback

* improve types with Pick

* refactor using type guard

* correct label

---------

Co-authored-by: Zachary Williams <ZachJW34@gmail.com>

* chore: typing error

* feat: scan for 3rd party ct plugins (#25749)

* chore: refactoring and types

* rework source of frameworks

* revert rename

* fix tests

* fix more tests

* types

* update code

* use same public API internally

* rename interfaces

* rename

* work on dev server api

* fix types

* fix test

* attempt to support getDevServerConfig

* tests

* add function to define framework [skip ci]

* rework a lot of types

* fix test

* update tests and types

* refactor

* revert changes

* lint

* fix test

* revert

* remove

* add "community" label [skip ci]

* refactor

* types

* lint

* fix bug

* update function name

* address feedback

* feat: scan for 3rd party ct plugins

* add e2e test

* unit tests [run ci]

* tweak resolution

* rebase, address comments

* fix windows paths

* remove .gitignore

* fix test

---------

Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>

* lint config

* spacing

* try fix race cond

* fix import error

* build binary

* try update snapshot

* try using require

* support namespaced definitions (#25804)

* remove category

* add icon prop

* support esm -> cjs compiled typescript

* fix test

* misc: add CTA footer to launchpad framework dropdown (#25831)

* remove test project dependencies

* rebase

* windows

* windows again

* add changelog entry

* changelog

* revert workflow

* remove worklfow

---------

Co-authored-by: Zachary Williams <ZachJW34@gmail.com>
Co-authored-by: Adam Stone-Lord <adams@cypress.io>

* chore: release @cypress/webpack-dev-server-v3.3.0

[skip ci]

* fix: Add missing error message when `req.continue` is used incorrectly (#25884)

---------

Co-authored-by: Adam Stone-Lord <adams@cypress.io>
Co-authored-by: Zachary Williams <ZachJW34@gmail.com>
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>
Co-authored-by: Matt Schile <mschile@cypress.io>
Co-authored-by: Alejandro Estrada <estrada9166@gmail.com>
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: Ryan Pei <ryanppei@gmail.com>
Co-authored-by: Emily Rohrbough <emilyrohrbough@yahoo.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: cypress-bot[bot] <47117332+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Mark Noonan <mark@cypress.io>
Co-authored-by: Stokes Player <stokes@cypress.io>
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: Zach Bloomquist <git@chary.us>
Co-authored-by: willmsC <50909991+willmsC@users.noreply.github.com>
Co-authored-by: Zach Bloomquist <github@chary.us>
Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Tim Griesser <tgriesser10@gmail.com>
Co-authored-by: Matt Henkes <mjhenkes@gmail.com>
Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net>
Co-authored-by: Ben M <benm@cypress.io>
Co-authored-by: Bill Glesias <bglesias@gmail.com>
Co-authored-by: Podles <78863563+podlesny-j@users.noreply.github.com>
Co-authored-by: Paolo Caleffi <p.caleffi@dreamonkey.com>
Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants