Should async tests use XCTestExpectation?

[paulyoung]

Related to Quick/Quick#132

I was wondering if there was a reason why toEventually, etc don't use XCTestExpectation internally via expectationWithDescription, waitForExpectationsWithTimeout, and fulfill.

[modocache]

+1, I think it'd be interesting to explore whether this is possible. Simply wrapping async functionality in XCTest might more stable, in the long run, than re-inventing that functionality in Nimble.

[paulyoung]

When considering test frameworks I've always thought it a plus when the default stuff was used under the hood.

[jeffh]

Last time I checked those methods work only if you're directly inside the test case, that would mean you would need to pass self (the test case instance) through nimble explicitly.

[paulyoung]

I can confirm that self.expectationWithDescription and self.waitForExpectationsWithTimeout is required to use those methods inside of a QuickSpec.

[jeffh]

That would require strange syntax to achieve properly in Nimble like:

expect(1).toEventually(equal(2), self)

Without some hackery or coordination with Quick (esp with shared examples).

I see using a implementation under the hood when appropriate. It can be a negative since it's tightly coupling to an external dependency. See Kiwi, Specta, and Cedar - which Xcode 6 test bundle support is in flux because of internal changes Apple has made. Using private APIs is a requirement if you want to do anything beyond what Apple conservatively exposes. I rather have something that doesn't support the latest wizbang feature from Apple instead of not working at all. But maybe my risk-factor is different from yours.

[jeffh]

I'm going to close this issue for now unless something changes in APIs or someone comes up with a good solution that doesn't require a major change in the general syntax. Using the public stuff would be great if possible.

[tonyxiao]

any updates 2 years later? :)

[modocache]

Nope! Same as before: expect(...).toEventually(...) provides a better API that isn't coupled to _XCTCurrentTestCase being set, or any of the other internal chicanery of XCTest. My original comment, in 2014, was wrong: rolling our own solution in Nimble has proven far more stable than interoperating with XCTest. Our async functionality continues to work, even on Linux, with no source code changes.

Thanks for the follow up, though! It's fun looking back on this decision, especially since I think we made the right one here. :)

[modocache]

@tonyxiao Is there something that you're missing from expect(...).toEventually(...), or some functionality that exists in XCTestExpectation but not in our API?

[tonyxiao]

@modocache not specifically. I liked the fact that XCTestExpectation seems explicit about promises being fulfilled whereas expect(...).toEventually(...) repeatedly checks value until expectation satisfies. Not sure how much difference that actually makes in every day usage.

[modocache]

XCTestCase's wait for expectations method does pretty much the same thing, waiting for any outstanding expectations to be fulfilled. It is nice that you can call it once to wait for all expectations. On the other hand, you can't wait for individual expectations. 
We could build an API that mirrors the XCTestExpectation pattern... but I prefer Nimble's individual expectation API, so I'm in no hurry to build something like that. :)

On Fri, Jul 1, 2016 at 5:42 PM -0400, "tonyxiao" notifications@github.com wrote:

@modocache not specifically. I liked the fact that XCTestExpectation seems explicit about promises being fulfilled whereas expect(...).toEventually(...) repeatedly checks value until expectation satisfies. Not sure how much difference that actually makes in every day usage.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[tonyxiao]

Got it. Thank you for the clarification!

[tonyxiao]

What about performance testing btw? Does Quick have something equivalent to measureBlock?

[iDevPro]

how to "expect(...).toEventually("ManyEquals here")" ?)