Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress 13 Hanging During Runs #27722

Closed
angiedaley opened this issue Aug 31, 2023 · 34 comments
Closed

Cypress 13 Hanging During Runs #27722

angiedaley opened this issue Aug 31, 2023 · 34 comments

Comments

@angiedaley
Copy link

Test code to reproduce

No code is failing. Code works when running in Desktop. Hangs between tests and stops running. No errors reported.

Cypress Mode

cypress run

Cypress Version

13.0.0

Browser Version

Chrome Version 116.0.5845.96 (Official Build) (64-bit)

Node version

v16.5.0

Operating System

Windows 11

Memory Debug Logs

No response

Other

We have 112 specs. Tests get to spec 55 and run the first two tests normally, then hangs and never progresses.

@cfuller1985
Copy link

I am having a similar issue that appears to have started yesterday where our CI/CD freezes when trying to upload results to Cypress Cloud. It's not happening on every spec but appears to be happening after the same spec. The spec nor config has changed recently. Seems odd that this started happening with the release of 13 which leads me to believe it's a bug with Cloud.

FWIW, We're on version 10 still and have never had this issue until yesterday.

@jennifer-shehane
Copy link
Member

jennifer-shehane commented Aug 31, 2023

@cfuller1985 @angiedaley Could you run Cypress in debug mode mode and print the entire set of logs here (extracting any private info)?

@funkadelic
Copy link

same. our cypress job has a timeout of 2 hours in Jenkins and when upgrading to 13 we are actually hitting this timeout, which never happened before.

will try to collect some debug logs

@egucciar
Copy link
Contributor

egucciar commented Aug 31, 2023

This is also happening to our team. I will try to get some logs. (Edit, didn't realize @funkadelic is also on our team 👋 , will coordinate on getting those logs)

@vincezipparro
Copy link

vincezipparro commented Sep 1, 2023

are any of your tests using selectFile? I have also had some hanging specifically when trying to upload video files > 200mbs. One of the errors from the logs I got was a FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

@nagash77
Copy link
Contributor

nagash77 commented Sep 1, 2023

Hi folks, we did make a change to make certain errors non-fatal in 13.1.0 which may help in some situations. Can folks please try upgrading to the latest version and see if that helps at all.

@joshuajtward
Copy link

I am also seeing this with 13.0.0 and 13.1.0. It seems to happen after certain tests, but also happens at inconsistent points in the test run, seems to crash before/while uploading the Test Replay, with DEBUG=cypress:server:*,cypress:https-proxy I am seeing the expected onMocha events:

 cypress:server:project onMocha end +1ms
  cypress:server:reporter got mocha event 'end' with args: [ undefined ] +1ms

  1 passing (8s)

  cypress:server:run received project end +11s
  cypress:server:run received videoController { videoController: undefined } +0ms
  cypress:server:run spec results: { error: null, reporter: 'spec', reporterStats: { suites: 0, tests: 1, passes: 1, pending: 0, failures: 0, start: 2023-09-01T15:02:59.932Z, end: 2023-09-01T15:03:08.138Z, duration: 8206 }, screenshots: [], spec: { absolute: '<filepath>/addToBag.test.ts', fileExtension: '.ts', fileName: 'addToBag', name: 'addToBag.test.ts', relative: '<filepath>/addToBag.test.ts' }, stats: { duration: 8211, endedAt: 2023-09-01T15:03:08.136Z, failures: 0, passes: 1, pending: 0, skipped: 0, startedAt: 2023-09-01T15:02:59.925Z, suites: 0, tests: 1 }, tests: [ { attempts: [Array], displayError: null, duration: 7263, state: 'passed', title: [Array] } ], video: null } +0ms
  cypress:server:run execute after:spec +0ms

And then I get this seemingly infinitely:

cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51198 } +616ms
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51519 } +5ms
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51448 } +90ms
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51529 } +3s
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51540 } +1s
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51537 } +0ms
 cypress:server:util:process_profiler current & mean memory and CPU usage by process group:
  cypress:server:util:process_profiler ┌─────────┬───────────────────┬──────────────┬─────────────────────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │       group       │ processCount │                pids                 │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────────────┼──────────────┼─────────────────────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │     'Chrome'      │      5       │ '92794, 92803, 92811, 92805, 92806' │    16.5    │     69.45      │  768.86  │    685.05    │   768.86    │
  cypress:server:util:process_profiler │    1    │     'cypress'     │      1       │               '91097'               │    2.6     │     26.03      │  369.88  │    322.07    │   369.88    │
  cypress:server:util:process_profiler │    2    │     'plugin'      │      1       │               '91120'               │     0      │      23.8      │  225.86  │    292.57    │   359.28    │
  cypress:server:util:process_profiler │    3    │ 'electron-shared' │      2       │           '91099, 91470'            │    0.7     │      0.45      │  55.61   │    55.38     │    55.61    │
  cypress:server:util:process_profiler │    4    │      'other'      │      1       │               '92834'               │     0      │       0        │   1.89   │     1.83     │    1.89     │
  cypress:server:util:process_profiler │    5    │      'TOTAL'      │      10      │                 '-'                 │    19.8    │      88.5      │ 1422.09  │   1012.57    │   1422.09   │
  cypress:server:util:process_profiler └─────────┴───────────────────┴──────────────┴─────────────────────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +10s

@vincezipparro
Copy link

@nagash77 still experiencing the same issues.

@sillicon
Copy link

sillicon commented Sep 1, 2023

I figured out that for my case, that it has something relates to test isolation.
My original tests were written with testIsolation disabled, and it takes 30+ minutes after test finish to let Cypress finish background processing and upload the test replay.
Then I re-write my tests with testIsolation enabled, it finishes test replay processing immediately.

@javgarr
Copy link

javgarr commented Sep 4, 2023

Also experiencing this after updating to Cypress 13. Rolling back.

@jan-trynda-crustlab
Copy link

Experiencing the same issue

I tried debugging it by running in a headed mode and I noticed, that my app redirects the whole page including the cypress runner and hangs then

Issue similar to: #23751

@sowardskimberly
Copy link

My team has been experiencing this as well with cypress 13.0.0 and now 13.1.0. Running it with our current settings with testIsolation: false, all of our parallelization runners will hang up. Sometimes 1 or 2 runners will last for a couple spec files, but then it will hang up. It will show the tests that passed in that spec file (3 tests passed ✔️ ), but it wont post/show the results in the table and appears hung up.

After reading this I ran it with testIsolation:true I was able to run all but 1 spec file that hung up.

Adding my info to add more data points:

Test code to reproduce
Code works when running in Desktop. Hangs between tests and stops running. No errors reported.
in my cypress.config.ts file I have testIsolation: false,

in my .yml file I have this content (leaving out some private .env materials)

jobs:
  cypress_run:
    name: 🧪 Cypress
    environment: Preview
    runs-on: windows-latest
    strategy:
      fail-fast: false
      matrix:
        # run 5 copies of the current job in parallel
        containers: [1, 2, 3, 4, 5]
    steps:
      - uses: actions/setup-node@v3
        with:
          node-version: 20.5.0

      - name: Checkout
        uses: actions/checkout@v3

      - name: Sleep for 10 seconds
        run: Start-Sleep -s 10
        shell: powershell

      - name: Cypress run
        uses: cypress-io/github-action@v6
        env:

Cypress Mode
cypress run

Cypress Version
13.0.0 and 13.1.0

Browser Version
default Electron browser

Node version
v20.5.0

Operating System
windows-latest

Memory Debug Logs
When running with testIsolation:true

cypress:protocol:capture:CSSManager Stylesheet changed but not tracked, not capturing +796ms
  cypress:protocol:capture:wi Insertion attempt for network:request-will-be-sent +1ms
  cypress:protocol:capture:wi Inserting network:request-will-be-sent into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:request-served-from-cache +1ms
  cypress:protocol:capture:wi Inserting network:request-served-from-cache into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:response-received +0ms
  cypress:protocol:capture:wi Inserting network:response-received into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:loading-finished +5ms
  cypress:protocol:capture:wi Inserting network:loading-finished into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:NetworkManager Network.loadingFinished {
  cypress:protocol:capture:NetworkManager   request: {
  cypress:protocol:capture:NetworkManager     requestInformation: {
  cypress:protocol:capture:NetworkManager       requestId: '4864.13749',
  cypress:protocol:capture:NetworkManager       requestURLs: [Array],
  cypress:protocol:capture:NetworkManager       type: 'Font',
  cypress:protocol:capture:NetworkManager       requestWillBeSentTimestamp: 451.535366,
  cypress:protocol:capture:NetworkManager       requestWillBeSentWallTime: 1693944784045.773,
  cypress:protocol:capture:NetworkManager       isCached: true
  cypress:protocol:capture:NetworkManager     },
  cypress:protocol:capture:NetworkManager     responseInformation: { mimeType: 'font/woff2', type: 'Font' },

When running with testIsolation:false the logs end at different times, but most commonly they freeze like the following. At the bottom of these logs I took out the hash numbers and put in *** because my team was not sure if that was a unique identifier or not:

  cypress:protocol:capture:wi Insertion attempt for network:response-received +1ms
  cypress:protocol:capture:wi Inserting network:response-received into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:loading-finished +1ms
  cypress:protocol:capture:wi Inserting network:loading-finished into events table with runnableId undefined, attempt undefined +1ms
  cypress:protocol:capture:NetworkManager Network.loadingFinished {
  cypress:protocol:capture:NetworkManager   request: {
  cypress:protocol:capture:NetworkManager     requestInformation: {
  cypress:protocol:capture:NetworkManager       requestId: '2768.11780',
  cypress:protocol:capture:NetworkManager       requestURLs: [Array],
  cypress:protocol:capture:NetworkManager       type: 'Font',
  cypress:protocol:capture:NetworkManager       requestWillBeSentTimestamp: 872.590465,
  cypress:protocol:capture:NetworkManager       requestWillBeSentWallTime: 1693938543412.073,
  cypress:protocol:capture:NetworkManager       isCached: true
  cypress:protocol:capture:NetworkManager     },
  cypress:protocol:capture:NetworkManager     responseInformation: { mimeType: 'font/woff2', type: 'Font' },
  cypress:protocol:capture:NetworkManager     responseFinishedInformation: {
  cypress:protocol:capture:NetworkManager       loadingFinishedTimestamp: 872.590524,
  cypress:protocol:capture:NetworkManager       runnableId: 'r3',
  cypress:protocol:capture:NetworkManager       attempt: 1
  cypress:protocol:capture:NetworkManager     }
  cypress:protocol:capture:NetworkManager   }
  cypress:protocol:capture:NetworkManager } +4ms
  cypress:protocol:capture:DOMManager DOM.childNodeRemoved { parentNodeId: 1811, nodeId: 1853 } +32ms
  cypress:protocol:capture:Fn Insertion attempt for dom:node-removed +32ms
  cypress:protocol:capture:Fn Inserting dom:node-removed into events table with runnableId r3, attempt 1 +0ms
  cypress:protocol:capture:wi Insertion attempt for network:request-will-be-sent +1ms
  cypress:protocol:capture:wi Inserting network:request-will-be-sent into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:request-served-from-cache +0ms
  cypress:protocol:capture:wi Inserting network:request-served-from-cache into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:response-received +1ms
  cypress:protocol:capture:wi Inserting network:response-received into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:loading-finished +1ms
  cypress:protocol:capture:wi Inserting network:loading-finished into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:NetworkManager Network.loadingFinished {
  cypress:protocol:capture:NetworkManager   request: {
  cypress:protocol:capture:NetworkManager     requestInformation: {
  cypress:protocol:capture:NetworkManager       requestId: '2768.11818',
  cypress:protocol:capture:NetworkManager       requestURLs: [Array],
  cypress:protocol:capture:NetworkManager       type: 'Font',
  cypress:protocol:capture:NetworkManager       requestWillBeSentTimestamp: 872.597416,
  cypress:protocol:capture:NetworkManager       requestWillBeSentWallTime: 1693938543419.031,
  cypress:protocol:capture:NetworkManager       isCached: true
  cypress:protocol:capture:NetworkManager     },
  cypress:protocol:capture:NetworkManager     responseInformation: { mimeType: 'font/woff2', type: 'Font' },
  cypress:protocol:capture:NetworkManager     responseFinishedInformation: {
  cypress:protocol:capture:NetworkManager       loadingFinishedTimestamp: 872.597511,
  cypress:protocol:capture:NetworkManager       runnableId: 'r3',
  cypress:protocol:capture:NetworkManager       attempt: 1
  cypress:protocol:capture:NetworkManager     }
  cypress:protocol:capture:NetworkManager   }
  cypress:protocol:capture:NetworkManager } +3ms
  cypress:protocol:capture:wi Insertion attempt for network:request-will-be-sent +1ms
  cypress:protocol:capture:wi Inserting network:request-will-be-sent into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:wi Insertion attempt for network:request-served-from-cache +18ms
  cypress:protocol:capture:wi Inserting network:request-served-from-cache into events table with runnableId undefined, attempt undefined +0ms
  cypress:protocol:capture:AssetManager writeToTarballFromFilePath indexed-tarball response {
  cypress:protocol:capture:AssetManager   err: undefined,
  cypress:protocol:capture:AssetManager   metadata: {
  cypress:protocol:capture:AssetManager     index: {
  cypress:protocol:capture:AssetManager       '***.gz': [Object],
  cypress:protocol:capture:AssetManager       '***.gz': [Object],
  cypress:protocol:capture:AssetManager       '***.gz': [Object],
  cypress:protocol:capture:AssetManager       '***.gz': [Object],
  cypress:protocol:capture:AssetManager       '***.gz': [Object],
 

@nagash77
Copy link
Contributor

nagash77 commented Sep 6, 2023

Hi everyone on this thread. Just wanted to drop an update here to keep everyone in the loop on what we are doing around this issue. Unfortunately, the logs provided right now don't have quite enough information for us to pinpoint the problem. We are working on adding some more debugging information into the logs that will hopefully get us closer to understanding and addressing this problem. We are working on that change which will be part of a Cypress Cloud release. Once that is released I will comment back here asking for folks to run their tests again and posts the logs with the added debugging information. Stay tuned.

@nagash77 nagash77 self-assigned this Sep 6, 2023
@nagash77
Copy link
Contributor

nagash77 commented Sep 6, 2023

Hi everyone! We have deployed our changes to logging. If folks could run their tests with DEBUG=cypress:protocol:capture,cypress-verbose:protocol:capture:tracked-cdp this will capture the new logging and hopefully help us diagnose the issue. Please post your logs back here when you have a moment. Quick guide for posting long things in a collapsable way

@billxlong
Copy link

For my case, it hangs after tests are completed

detailed logging The log is very repetitive, after
  6 passing (56s)

[mochawesome] Report JSON saved to /Users/XXXX/xxx/cypress/results/.jsons/mochawesome.json

The logs start to see repetitive stuff like this

  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.responseReceivedExtraInfo +199ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.responseReceived +1ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.loadingFinished +1ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +1ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +5ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +2ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.requestWillBeSent +925ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.requestWillBeSentExtraInfo +2ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +2ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +1ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +4ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +2ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.responseReceivedExtraInfo +197ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.responseReceived +3ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.loadingFinished +1ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +2ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining DOM.attributeModified +5ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Runtime.bindingCalled +3ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.requestWillBeSent +6s
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.requestWillBeSentExtraInfo +4ms

Nothing else was seen and starts hanging.

@ryanthemanuel
Copy link
Collaborator

ryanthemanuel commented Sep 7, 2023

Thanks @billxlong, can you share the last log that is prefixed with cypress:protocol:capture? (And maybe the surrounding logs)

@billxlong
Copy link

billxlong commented Sep 7, 2023

Thanks @billxlong, can you share the last log that is prefixed with cypress:protocol:capture? (And maybe the surrounding logs)

There you go:

detailed logging
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress:protocol:capture afterTest: draining cdp client, allowSpilloverEvents: true +708ms
  cypress:protocol:capture afterTest: flushing pending timestamp sync events +0ms
  cypress:protocol:capture afterTest: done +0ms
  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining DOM.attributeModified +5ms
  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining Runtime.bindingCalled +1ms
  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining Runtime.bindingCalled +0ms
  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining Runtime.bindingCalled +0ms

  6 passing (51s)

  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining Runtime.bindingCalled +19ms
  cypress-verbose:protocol:capture:tracked-cdp handling spillover event while draining Runtime.bindingCalled +0ms
[mochawesome] Report JSON saved to /Users/XXXXX/cypress/results/.jsons/mochawesome.json

  cypress:protocol:capture afterSpec: draining cdp client +1s
  cypress:protocol:capture afterSpec: resolving network body promises +0ms
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.requestWillBeSent +3s
  cypress-verbose:protocol:capture:tracked-cdp skipping event while draining Network.requestWillBeSentExtraInfo +2ms

FWIW I'm running this locally with my Mac not in CI @ryanthemanuel

@sowardskimberly
Copy link

Here is the logging for my run that is hung up in the ci/cd pipeline

detailed logs
 cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +6s
PowershellResult { stdout: '', stderr: '', code: 0 }
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +2ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +4ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +19ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +2ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +73ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Network.requestWillBeSent, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Network.requestWillBeSent, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for DOM.attributeModified, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for DOM.attributeModified, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Network.requestWillBeSentExtraInfo, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Network.requestWillBeSentExtraInfo, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Network.responseReceivedExtraInfo, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Network.responseReceivedExtraInfo, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Network.responseReceived, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Network.responseReceived, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Network.loadingFailed, pending callbacks: 1 +7ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Network.loadingFailed, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +3ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +0ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms
  cypress-verbose:protocol:capture:tracked-cdp adding outstanding callback for Runtime.bindingCalled, pending callbacks: 1 +1ms
  cypress-verbose:protocol:capture:tracked-cdp removing outstanding callback for Runtime.bindingCalled, pending callbacks: 0 +0ms

@ryanthemanuel
Copy link
Collaborator

Thanks @sowardskimberly can you share the last log that is prefixed with cypress:protocol:capture? (And maybe the surrounding logs)

@sowardskimberly
Copy link

sowardskimberly commented Sep 7, 2023

@ryanthemanuel what I have shared is where my logs end. My test has been hung up for over 1 hour since the last test passed. In the logs I pasted, I included where it passed, and then the end where it is waiting. I assume you mean cypress-verbose:protocol:capture:tracked-cdp because i'm not seeing a log for cypress:protocol:capture.

image

image

@ryanthemanuel
Copy link
Collaborator

Hello everyone. We just released something in the cloud that will hopefully help some of the hangings that people were seeing between specs. I'm not sure this covers everyone's issue, but I'm curious to see if it has helped anyone so far. We have a few additional fixes we are looking into as well.

@muratkeremozcan
Copy link

muratkeremozcan commented Sep 8, 2023

For me, this can be reproduced locally,

I commented on a potentially duplicate issue.

Has an open repo, and repro steps anyone can try, as well as a public Cy Cloud project.

#27769 (comment)

@cbookg
Copy link

cbookg commented Sep 8, 2023

Hello everyone. We just released something in the cloud that will hopefully help some of the hangings that people were seeing between specs. I'm not sure this covers everyone's issue, but I'm curious to see if it has helped anyone so far. We have a few additional fixes we are looking into as well.

Resolved the issue for me.

@maisano-patreon
Copy link

Hello everyone. We just released something in the cloud that will hopefully help some of the hangings that people were seeing between specs. I'm not sure this covers everyone's issue, but I'm curious to see if it has helped anyone so far. We have a few additional fixes we are looking into as well.

This also resolved the issue for me.

@JakeBradenPro
Copy link

Hello everyone. We just released something in the cloud that will hopefully help some of the hangings that people were seeing between specs. I'm not sure this covers everyone's issue, but I'm curious to see if it has helped anyone so far. We have a few additional fixes we are looking into as well.

Resolved for us

@javgarr
Copy link

javgarr commented Sep 11, 2023

Seems this would have solved the issue on our side as well, thanks!

@joshuajtward
Copy link

I am also seeing this with 13.0.0 and 13.1.0. It seems to happen after certain tests, but also happens at inconsistent points in the test run, seems to crash before/while uploading the Test Replay, with DEBUG=cypress:server:*,cypress:https-proxy I am seeing the expected onMocha events:

 cypress:server:project onMocha end +1ms
  cypress:server:reporter got mocha event 'end' with args: [ undefined ] +1ms

  1 passing (8s)

  cypress:server:run received project end +11s
  cypress:server:run received videoController { videoController: undefined } +0ms
  cypress:server:run spec results: { error: null, reporter: 'spec', reporterStats: { suites: 0, tests: 1, passes: 1, pending: 0, failures: 0, start: 2023-09-01T15:02:59.932Z, end: 2023-09-01T15:03:08.138Z, duration: 8206 }, screenshots: [], spec: { absolute: '<filepath>/addToBag.test.ts', fileExtension: '.ts', fileName: 'addToBag', name: 'addToBag.test.ts', relative: '<filepath>/addToBag.test.ts' }, stats: { duration: 8211, endedAt: 2023-09-01T15:03:08.136Z, failures: 0, passes: 1, pending: 0, skipped: 0, startedAt: 2023-09-01T15:02:59.925Z, suites: 0, tests: 1 }, tests: [ { attempts: [Array], displayError: null, duration: 7263, state: 'passed', title: [Array] } ], video: null } +0ms
  cypress:server:run execute after:spec +0ms

And then I get this seemingly infinitely:

cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51198 } +616ms
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51519 } +5ms
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51448 } +90ms
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51529 } +3s
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51540 } +1s
  cypress:server:util:socket_allowed allowed socket closed, removing { localPort: 51537 } +0ms
 cypress:server:util:process_profiler current & mean memory and CPU usage by process group:
  cypress:server:util:process_profiler ┌─────────┬───────────────────┬──────────────┬─────────────────────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │       group       │ processCount │                pids                 │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────────────┼──────────────┼─────────────────────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │     'Chrome'      │      5       │ '92794, 92803, 92811, 92805, 92806' │    16.5    │     69.45      │  768.86  │    685.05    │   768.86    │
  cypress:server:util:process_profiler │    1    │     'cypress'     │      1       │               '91097'               │    2.6     │     26.03      │  369.88  │    322.07    │   369.88    │
  cypress:server:util:process_profiler │    2    │     'plugin'      │      1       │               '91120'               │     0      │      23.8      │  225.86  │    292.57    │   359.28    │
  cypress:server:util:process_profiler │    3    │ 'electron-shared' │      2       │           '91099, 91470'            │    0.7     │      0.45      │  55.61   │    55.38     │    55.61    │
  cypress:server:util:process_profiler │    4    │      'other'      │      1       │               '92834'               │     0      │       0        │   1.89   │     1.83     │    1.89     │
  cypress:server:util:process_profiler │    5    │      'TOTAL'      │      10      │                 '-'                 │    19.8    │      88.5      │ 1422.09  │   1012.57    │   1422.09   │
  cypress:server:util:process_profiler └─────────┴───────────────────┴──────────────┴─────────────────────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +10s

This appears to have resolved the issue for us too, thanks! 🙌

@nagash77
Copy link
Contributor

Hi @angiedaley have you been able to try using 13.1.0 to see if your issues are resolved?

@ghost
Copy link

ghost commented Sep 12, 2023

hi all, can confirm that it happens in Cypress 13.1.0 when using selectFile with a big file (we have a 117MB one)

@vincezipparro
Copy link

Confirmed it is still happening in 13.2.0 as well using selectFile

@jennifer-shehane
Copy link
Member

Closing this issue as most of this occurrence has been resolved. If you’re still encountering hanging in 13.x, please open a new issue detailing the exact error, debug logs, and reproducible example so we can track down the new situation that may be causing hanging in your specific situation.

@angiedaley
Copy link
Author

@nagash77 Unfortunately this does not resolve our issue. I was out of the office for two weeks, apologies for not commenting sooner. I'll work with support.

@jennifer-shehane
Copy link
Member

Hey @angiedaley, sorry to hear this didn't resolve your issue. There are a lot of inputs that could influence hanging. Please reach out to support providing more details so we can focus it down more on what separates your issue from the other commenters.

@bcata11
Copy link

bcata11 commented May 21, 2024

Have the same problem, a test remains hanging when we are waiting for an apicall. Is there a method to fail the test or the pipeline if this happens?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests