Spec hangs when error thrown in test:after:run listener

[oleksiydubovykTR]

Related to #8363, basically the same issue.

I've been waiting for the fix for quite a long time, and kind of disappointed to see that it seems that it still hangs for the cy.get().should() assertions, in both cy run and cy open, in both 5.2.0 and 5.3.0.

cy run

cy open

I obviously just changed the expected Log Out button text to Log OutZZZ, and that's it, the test will hang for 4 hours in your CI, and will never retry, or skip to other tests.

[bahmutov]

Can you provide a reproducible example, we believe we have fixed it in #8363 but seems something else in your particular situation causes a problem

[NurmemetAbliz]

I have the same issue . My tests in CI nightly build are normally completed around 2 hours . But it is still running even after 8-9 hours recently for the same number of tests and the same test execution environment , and it looks like never ending . So currently I am having no choice but to cancel the tests .

[oleksiydubovykTR]

Can you provide a reproducible example, we believe we have fixed it in #8363 but seems something else in your particular situation causes a problem

@bahmutov
Hi Gleb, yep, you're right. I managed to create a repro:

https://github.com/dubbha/cypress-test-tiny/tree/issue8701%405.3.0
https://github.com/cypress-io/cypress-test-tiny/compare/master...dubbha:issue8701@5.3.0

And while trying to create it I realized it all boils down to a script thowing within a Cypress.on('test:after:run'):

Cypress.on('test:after:run', (test) => {
  if (test.state === 'failed') {
    throw new Error('I am throwing')
  }
})

Originally this was adding screenshots to mochawesome reports for the failed tests.
In my case it throws because it was still trying to read env.RETRIES from cypress.json, but it could throw for any reason.
Wrapping it in a simple try/catch does help.

Also it doesn't even matter if retries are configured in cypress.json or not.

So I am not sure if it can still be considered a bug or just a thing to be aware of for the guys out there.

[bahmutov]

Yeah I can recreate the problem in https://github.com/cypress-io/cypress-test-tiny/tree/throw-after

/// <reference types="cypress" />
Cypress.on('test:after:run', (test) => {
  if (test.state === 'failed') {
    // https://github.com/cypress-io/cypress/issues/8701
    throw new Error('I am throwing')
  }
})

describe('page', () => {
  it('works', () => {
    cy.visit('https://example.cypress.io')
    cy.get('[class^="main-nav-link active"]').should('contain', 'ZZZ')
  })
})

The test fails, throws an error and keeps on spinning

By the way, even in a successful test where you throw from test:after:run does not finish

Cypress.on('test:after:run', (test) => {
  throw new Error('I am throwing')
})

describe('page', () => {
  it('works', () => {
    cy.visit('https://example.cypress.io')
  })
})

[bahmutov]

I have not verified what happens if an error is thrown in other event callbacks

[kuceb]

Looks like this has always causes the spec to hang, even before retries. I'll take a look. Thanks for the reproducible

[jennifer-shehane]

Duplicate of #2271

[cypress-bot]

The code for this is done in cypress-io/cypress#8709, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[cypress-bot]

Released in 5.4.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v5.4.0, please open a new issue.