New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception in root level after() doesn't fail test run in interactive mode #2296

Open
amitzur opened this Issue Aug 7, 2018 · 9 comments

Comments

4 participants
@amitzur

amitzur commented Aug 7, 2018

Current behavior:

If there's an after() outside of any describe, it or context ("rool level"), which throws an exception, that doesn't fail the test run. This is different from how it works in mocha, where you get a "after all" hook failure.

Also, if it's a synchronous failure, the exception doesn't show up anywhere.
If it's async, it shows up like this:

screen shot 2018-08-07 at 5 13 10 pm

Desired behavior:

The whole test run should fail (sometimes you would want to place an assertion there that is possible only after all tests have run).

Steps to reproduce:

Add this to your tests:

after(() => {
  cy.then(() => {
    throw new Error('bla bla');
  });
});

Versions

Cypress: 3.0.3
OS: Mac OS X 10.12.6 (Sierra)
Browser: both Electron (59) and Chrome (68)

@amitzur amitzur changed the title from Exception in root level after() doesn't fail test run to Exception in root level after() doesn't fail test run in interactive mode Aug 8, 2018

@amitzur

This comment has been minimized.

Show comment
Hide comment
@amitzur

amitzur Aug 8, 2018

I just found out that running cypress run or through the module API does report a failed test. It's just cypress open that shows the test as green.

amitzur commented Aug 8, 2018

I just found out that running cypress run or through the module API does report a failed test. It's just cypress open that shows the test as green.

@jennifer-shehane

This comment has been minimized.

Show comment
Hide comment
@jennifer-shehane

jennifer-shehane Aug 8, 2018

Member

Yes, this is likely a UI level issue if that is the case.

Member

jennifer-shehane commented Aug 8, 2018

Yes, this is likely a UI level issue if that is the case.

@amitzur

This comment has been minimized.

Show comment
Hide comment
@amitzur

amitzur Aug 16, 2018

Hi there, is there progress in investigation of this issue?

amitzur commented Aug 16, 2018

Hi there, is there progress in investigation of this issue?

@Bkucera

This comment has been minimized.

Show comment
Hide comment
@Bkucera

Bkucera Aug 16, 2018

Member

@amitzur considering this is only a UI bug, it's a pretty low priority at the moment, but PRs are welcome!

Member

Bkucera commented Aug 16, 2018

@amitzur considering this is only a UI bug, it's a pretty low priority at the moment, but PRs are welcome!

@amitzur

This comment has been minimized.

Show comment
Hide comment
@amitzur

amitzur Aug 16, 2018

glad to take the challenge. Any chance to get a head start on code? I already got familiar with it, but where are test results communicated to the client?

amitzur commented Aug 16, 2018

glad to take the challenge. Any chance to get a head start on code? I already got familiar with it, but where are test results communicated to the client?

@jennifer-shehane

This comment has been minimized.

Show comment
Hide comment
@jennifer-shehane

jennifer-shehane Aug 16, 2018

Member

@amitzur The code for this is in our reporter package which is a React app. I'm not totally sure where this discrepancy is at - but here's a link to the runnable area of the app (may not be where the error is), but hopefully gets you started. https://github.com/cypress-io/cypress/blob/develop/packages/reporter/src/runnables/runnable-and-suite.jsx#L8

Member

jennifer-shehane commented Aug 16, 2018

@amitzur The code for this is in our reporter package which is a React app. I'm not totally sure where this discrepancy is at - but here's a link to the runnable area of the app (may not be where the error is), but hopefully gets you started. https://github.com/cypress-io/cypress/blob/develop/packages/reporter/src/runnables/runnable-and-suite.jsx#L8

@amitzur

This comment has been minimized.

Show comment
Hide comment
@amitzur

amitzur Aug 16, 2018

great, thanks!
2 things:

  1. The URL you listed is from the whitelist branch, it needs to be fixed to master or develop.
  2. How do you setup React devtools? Via a browser extension or a standalone app? I couldn't easily find how to inject <script src="http://localhost:8097"></script> if it's a standalone app, or how to provide a path to the extension via a plugin.

amitzur commented Aug 16, 2018

great, thanks!
2 things:

  1. The URL you listed is from the whitelist branch, it needs to be fixed to master or develop.
  2. How do you setup React devtools? Via a browser extension or a standalone app? I couldn't easily find how to inject <script src="http://localhost:8097"></script> if it's a standalone app, or how to provide a path to the extension via a plugin.
@jennifer-shehane

This comment has been minimized.

Show comment
Hide comment
@jennifer-shehane

jennifer-shehane Aug 16, 2018

Member
  1. Oops, yeah, should be PR from develop
  2. @chrisbreiding
Member

jennifer-shehane commented Aug 16, 2018

  1. Oops, yeah, should be PR from develop
  2. @chrisbreiding
@chrisbreiding

This comment has been minimized.

Show comment
Hide comment
@chrisbreiding

chrisbreiding Aug 16, 2018

Collaborator

If you run in Chrome, you should be able to install the extension the standard way by visiting https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi

Collaborator

chrisbreiding commented Aug 16, 2018

If you run in Chrome, you should be able to install the extension the standard way by visiting https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment