Test fails: "expected to eventually be true, got <true>"

[jorishjanssen]

Nimble: 5.1.0, xcode: 8.0 (8A218a), using cocoapods, UI Testing bundle, iOS simulator (iPad 2 Air)

I really like Nimble, thanks!

Observed behaviour:
Tests sometimes fail with the following message: "expected to eventually be true, got ". When rerunning the same test it often just passes.

Expected behaviour:
When expectations gets true using the matcher beTrue() it should not fail.

Steps to reproduce:

• I haven't been able to consistently reproduce. The same tests sometimes fail with this message and sometimes don't. It happens in different test classes.
• The code is always of the following form:
  expect(myFunc()).toEventually(beTrue(), timeout: myTimeout) where myFunc() is in a different helper class.
• The same happens with the beTruthy() matcher.

Screenshot from Xcode

[jeffh]

Hey @jorishjanssen,

Thanks for filing a bug. The behavior you see is definitely not ideal. It seems like two separate issues.

First, Nimble is producing an error message that is inconsistent with the actual value it produced. The error message Nimble produces is inconsistent with the result the matcher got because the failure message string currently calls myFunc() again to get the value – which happens to change to true the split-second after the matcher failed.

The intermittent fails depends a lot of what myFunc() does. One way to check is by trying to increase your timeout duration. But it's difficult to say without knowing what myFunc() is actually doing. Unfortunately, this is inherent to asynchronous tests.

[jorishjanssen]

Hey @jeffh

Thanks for the response. The explanation about the error message inconsistency clarified a lot. I changed the timeout and it seems to have solved the issue. Nonetheless, would still be a nice improvement to avoid the second call to myFunc() to get the failure message string for others who are not aware of this.

Thanks again for the help!

Joris