Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle skipped tests in TAP reporter #262

Merged
merged 5 commits into from Sep 29, 2015
Merged

Conversation

@nealrichardson
Copy link
Contributor

@nealrichardson nealrichardson commented May 26, 2015

Because skipped tests did not have result$passed, they were marked as "not ok" in the TAP output. Following http://testanything.org/tap-specification.html, I added a skip directive to the output in case of result$skipped.

I'd happily write a test for this change if you could indicate where that should be. I didn't find any tests for the reporter output, only for the existence of reporters. I did confirm that the output with the skip directives is correctly parsed, at least by the Jenkins TAP parser:

screen shot 2015-05-26 at 11 35 27 am

@coveralls
Copy link

@coveralls coveralls commented May 26, 2015

Coverage Status

Coverage decreased (-0.89%) to 80.18% when pulling a76cb08 on nealrichardson:master into 4d404c1 on hadley:master.

@nealrichardson
Copy link
Contributor Author

@nealrichardson nealrichardson commented May 27, 2015

Ok, tests for the TAP reporter output added, including a test for the skip directive. This should increase test coverage, though the coveralls job in https://travis-ci.org/hadley/testthat/builds/64178838 timed out (which appears to be why the Travis build was marked as failing; the tests passed).

test_that("TAP reporter handles context and pass/fail/skip", {
filename <- tempfile(fileext="tap")
sink(filename)
test_dir("test_dir", reporter = "tap")

This comment has been minimized.

@hadley

hadley Sep 24, 2015
Member

I think you could simplify this with capture.output()

This comment has been minimized.

@nealrichardson

nealrichardson Sep 28, 2015
Author Contributor

Yes, thanks, much nicer.

@@ -0,0 +1,6 @@
context("Skip")

This comment has been minimized.

@hadley

hadley Sep 24, 2015
Member

Could you please use a lower case skip to be consistent with the other files?

This comment has been minimized.

@nealrichardson

nealrichardson Sep 28, 2015
Author Contributor

Done, though I noticed that "Bare" and "Basic" in test_dir are also capitalized.

@hadley
Copy link
Member

@hadley hadley commented Sep 28, 2015

Could you please also add a bullet point to NEWS?

@hadley hadley merged commit 2517adc into r-lib:master Sep 29, 2015
2 checks passed
2 checks passed
continuous-integration/appveyor AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@hadley
Copy link
Member

@hadley hadley commented Sep 29, 2015

Thanks! (For future reference, I don't get a notification when you make changes, just when you make comments, so once you're done, it's helpful to add a comment like "PTAL")

@nealrichardson
Copy link
Contributor Author

@nealrichardson nealrichardson commented Sep 29, 2015

Ah ok, thanks for pointing that out, will do.

@nealrichardson
Copy link
Contributor Author

@nealrichardson nealrichardson commented Sep 29, 2015

Looks like the NEWS bullet I added was clobbered in a merge: nealrichardson/testthat@master...hadley:masterdiff-8312ad0561ef661716b48d09478362f3L10

I could make a new PR with that if you want, but maybe it's easier for you to just add it back.

hadley added a commit that referenced this pull request Sep 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants