Error in before() block is ignored if RETRIES is greater than zero

[drewbrend]

In our case: we request some test data in a before() block using cy.request(), then use that data in our tests. If the request to get the data fails the test is guaranteed to fail so it should not execute. When we use the retries plugin the test does execute, then shows the error that occurs because we don't have the test data. The actual error causing this is that the service we were reaching out to returned a 404 but the error shown in the test report is that cypress timed out trying to find an element that didn't exist.

This masks the real issue that causes a test failure and makes debugging failures in CI very difficult.

To repro this issue, run the spec from the branch in my fork of this repo: https://github.com/drewbrend/cypress-plugin-retries/blob/patch-1/cypress/integration/spec.js

Expected: If a failure in the before() or beforeEach() block happens, the test shouldn't be ran, regardless of the number of retries.

Actual: If the value of the RETRIES environment variable is greater than zero, even if a failure occurs in a before() or beforeEach() block, the test will be run. This causes the error shown in the test results to be inaccurate.

Here is a screenshot of what this looks like after it is ran:

[kuceb]

thanks @drewbrend, I will take a look at this

[kuceb]

🎉 This issue has been resolved in version 1.3.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀