All(?) timeouts report as "cy.then() timed out after waiting '4000ms'" rather than printing actual expectation failure

[jagregory]

• cypress-testing-library version: 5.0.1
• node version: 12.6.0
• npm (or yarn) version: 1.17.3

Since upgrading to 5.0.1, it seems any timeouts are all being reported as Your callback function returned a promise which never resolved.

Any expectation that times out now just prints out the below, which obviously makes it pretty difficult to know which assertion it's actually failing on.

     CypressError: cy.then() timed out after waiting '4000ms'.

Your callback function returned a promise which never resolved.

The callback function was:

thenArgs => {
        const getValue = () => {
          const value = commandImpl(thenArgs.document);
          const result = Cypress.$(value); // Overriding the selector of the jquery object because it's displayed in the long message of .should('exist') failure message
          // Hopefully it makes it clearer, because I find the normal response of "Expected to find element '', but never found it" confusing

          result.selector = `${queryName}(${queryArgument(args)})`;

          if (result.length > 0) {
            consoleProps.yielded = result.toArray();
          }

          return result;
        };

        const resolveValue = () => {
          // retry calling "getValue" until following assertions pass or this command times out
          return Cypress.Promise.try(getValue).then(value => {
            return cy.verifyUpcomingAssertions(value, waitOptions, {
              onRetry: resolveValue
            });
          });
        };

        if (queryRegex.test(queryName)) {
          // For get* queries, do not retry
          return getValue();
        }

        return resolveValue().then(subject => {
          // Remove the error that occurred because it is irrelevant now
          if (consoleProps.error) {
            delete consoleProps.error;
          }

          return subject;
        });
      }

[jagregory]

Interestingly, all the failures also report that they timed out after waiting only 4000ms despite my tests specifying custom timeouts for them all.

Each one of those failed with a CypressError: cy.then() timed out after waiting '4000ms'. message.

[jagregory]

After a bit of digging, it looks like maybe ab5263d introduced a bug.

Previously there was this code:

.window({log: false})
.then({timeout: waitOptions.timeout + 100}, thenHandler)

but the then has been replaced and doesn't retain the timeout option.

Tentative testing with the timeout re-added seems to have fixed the issue of timeouts not being respected (see: jagregory@d7fe43e). I'm unsure whether this also fixes the incorrect failure messages (I assume that's what the + 100 is for)

[jagregory]

I can confirm that with the above mentioned change, if a legitimate timeout occurs I now get the expected message. For example: CypressError: Timed out retrying: Expected to find element: '#option-1', but never found it.

[kentcdodds]

🎉 This issue has been resolved in version 5.0.2 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[Jmaharman]

We did have tests to check the timeouts were adhered to, I imagine a scenario was being missed.

@jagregory would you be able to add a test which shows the initial failure? It would be good to have a test in the suite to confirm this doesn't get broken again.