-
-
Notifications
You must be signed in to change notification settings - Fork 910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better pending
test case output
#770
Comments
Hi, Should xit, xdescribe, ... appear as "ignored" in the test results in AppCode? When I set xit, xdescribe, ... they never appear to the test results. I think will be good that they appear as ignored so we know we have pending tests to implement. They don't appear either in Xcode. |
This is related to #490, specifically to #490 (comment). |
Does anyone still desire this feature? I have a PR related to it (#771), but resolving conflicts and merging it might be some significant work, and I'd rather focus my free time's energy on other issues if this one is not a high priority. |
@jwfriese I would love for pending tests to be visible. I always spec out the expected behaviour with pending For me, pending tests are sort-of a todo list written in test files. Also, a CI environment cannot fail because of ignored tests if it does not know about the ignored tests. |
Hello guys, I’m working in AppCode, and we are very excited to add support for Quick framework to our IDE soon. But we encountered several problems during implementation of pending test cases support. Essentially it’s not very convenient for us to parse ‘pending’ test cases because pending() produces pretty ambiguous output. Here is an example:
Xcode console output:
Pending: it
Pending: it
Pending: it
Test Suite 'Selected tests' started at 2018-02-06 17:18:08.129
Test Suite 'Quick - macOSTests.xctest' started at 2018-02-06 17:18:08.135
Test Suite 'PendingTestsEx' started at 2018-02-06 17:18:08.136
Test Case '-[Quick___macOSTests.PendingTestsEx context__it]' started.
Test Case '-[Quick___macOSTests.PendingTestsEx context__it]' passed (0.985 seconds).
Test Case '-[Quick___macOSTests.PendingTestsEx context__it_2]' started.
Test Case '-[Quick___macOSTests.PendingTestsEx context__it_2]' passed (0.005 seconds).
Test Suite 'PendingTestsEx' passed at 2018-02-06 17:18:09.129.
Executed 2 tests, with 0 failures (0 unexpected) in 0.990 (0.993) seconds
Test Suite 'Quick - macOSTests.xctest' passed at 2018-02-06 17:18:09.131.
Executed 2 tests, with 0 failures (0 unexpected) in 0.990 (0.996) seconds
Test Suite 'Selected tests' passed at 2018-02-06 17:18:09.132.
Executed 2 tests, with 0 failures (0 unexpected) in 0.990 (1.003) seconds
Program ended with exit code: 0
As you can see, It's unclear, which one of 'it' is pending. For 'pending' cases I'd love to see test id's as for usual tests. Probably output may be something like (now we can identify where pending tests are, and put them into test tree in right place):
Pending: Quick___macOSTests.PendingTestsEx it
Pending: Quick___macOSTests.PendingTestsEx context__it
Pending: Quick___macOSTests.PendingTestsEx context__it_3
Test Suite 'Selected tests' started at 2018-02-06 17:18:08.129
Test Suite 'Quick - macOSTests.xctest' started at 2018-02-06 17:18:08.135
Test Suite 'PendingTestsEx' started at 2018-02-06 17:18:08.136
Test Case '-[Quick___macOSTests.PendingTestsEx context__it_1]' started.
Test Case '-[Quick___macOSTests.PendingTestsEx context__it_1]' passed (0.985 seconds).
Test Case '-[Quick___macOSTests.PendingTestsEx context__it_2]' started.
Test Case '-[Quick___macOSTests.PendingTestsEx context__it_2]' passed (0.005 seconds).
Test Suite 'PendingTestsEx' passed at 2018-02-06 17:18:09.129.
Executed 2 tests, with 0 failures (0 unexpected) in 0.990 (0.993) seconds
Test Suite 'Quick - macOSTests.xctest' passed at 2018-02-06 17:18:09.131.
Executed 2 tests, with 0 failures (0 unexpected) in 0.990 (0.996) seconds
Test Suite 'Selected tests' passed at 2018-02-06 17:18:09.132.
Executed 2 tests, with 0 failures (0 unexpected) in 0.990 (1.003) seconds
Program ended with exit code: 0
Also if it's possible, we'd love to see similar behaviour for xit, xdescribe, xcontext, xitBehavesLike (probably enabled by additional knob like environmental var). To sum up: all we need is to have an opportunity to identify pending test cases in the test tree by unique id.
Thank you in advance!
Environment:
The text was updated successfully, but these errors were encountered: