Screenshot doesn't always capture error / failure details

[andreiashu]

Current behavior:

We are using Cypress for testing an API E2E. The screenshots will explain better for me but basically when the test fails the screenshot does not wait long enough in order for the cypress UI to update the DOM with the error details.

Desired behavior:

Screenshots should contain error details

How to reproduce:

I just run the following test with cypress run basically:

describe('demo a failing test', () => {
  it('will fail!', function() {
    expect({"foo": "bar"}).to.deep.equal("foo": "baz");
  })
});

Additional Info (images, stack traces, etc)

Current:

How it appears when the next test fails:

• Operating System: OSx and Linux
• Cypress Version: 2.0.2
• Browser Version: Canary 66

[brian-mann]

This is a known issue for awhile. It has to do with React (which is what we built the reporter in).

The problem is that taking a screenshot is an async process. But React also renders async. To date - we acutally do synchronize taking a screenshot with React - but its in a shallow component and there is a chance the nested components have not rendered yet.

The solution seems simple - wait until all the the UI parts have rendered right?

The problem with that is that the longer we wait to take the screenshot the less accurate it will be when the state of your app fails. We're talking about milliseconds here - but those matter.

The question is - would you rather have an accurate screenshot of the way your app looked when it failed? Or would you rather have a screenshot including the error in our reporter? It's unfortunately not possible to have both.

[brian-mann]

The other thing worth mentioning here - is that this is why we capture both the error + stack, the screenshot, and the video in our dashboard.

It's not enough to have just 1 of them - you need more context to really understand what's going on.

[chrisbreiding]

We're working on reproducing this and having trouble. From your screenshot, it looks like there are more tests than in your example code. Could you post all the code you used to reproduce this?

[jennifer-shehane]

Hey, I know this does occur as I have seen it myself. We'll need a reproducible example to track down the exact issue though. Can anyone provide?

[jennifer-shehane]

Unfortunately we have to close this issue as there is not enough information to reproduce the problem.

Please comment in this issue with a reproducible example and we will reopen the issue. 🙏