JUnitXMLReportPlugin in JUnitXmlTestsListener.scala is not adding skipped attribute to the testsuite root element #2198

Closed
ppadial opened this Issue Sep 4, 2015 · 4 comments

Comments

Projects
None yet
3 participants
@ppadial

ppadial commented Sep 4, 2015

Hi,

JUnitXML document in testsuite root element could contains skipped="numberofskippedtests" so other plugins can read this info (like jenkins) and get how many skipped tests are.

Looking into the (http://www.scala-sbt.org/release/sxr/sbt/JUnitXmlTestsListener.scala.html) this attribute is not being added
val result = <testsuite hostname={hostname} name={name}
tests={tests + ""} errors={errors + ""} failures={failures + ""}
time={(duration/1000.0).toString} >

So, could we add this attribute?

@eed3si9n

This comment has been minimized.

Show comment
Hide comment
@eed3si9n

eed3si9n Sep 4, 2015

Member

Sure. Send us a pull request.

Member

eed3si9n commented Sep 4, 2015

Sure. Send us a pull request.

@eed3si9n eed3si9n added the Enhancement label Sep 4, 2015

@ashleymercer

This comment has been minimized.

Show comment
Hide comment
@ashleymercer

ashleymercer Nov 29, 2016

I'd like to have a go at picking this up since it's currently causing me pain. A couple of questions:

  • should I be working against 0.13 or the 1.0 branch?
  • the approach taken by Ant is documented on this bug report - I need to read more background but it seems there's some incompatibility between XML reports in Junit 3 and 4. Will report back here once I've worked out what's going on, but in the meantime can we assume Junit 4 compatiblity in sbt (and, presumably, downstream tools?)
  • an alternative strategy would be to delegate to the specific test library to handle this. Specs2 and scalatest already both have support for outputting Junit XML so it would remove some duplication - although I might need some help getting this to work correctly, and we'd need to decide what to do if a different (non-specs/non-scalatest) framework is being used

ashleymercer commented Nov 29, 2016

I'd like to have a go at picking this up since it's currently causing me pain. A couple of questions:

  • should I be working against 0.13 or the 1.0 branch?
  • the approach taken by Ant is documented on this bug report - I need to read more background but it seems there's some incompatibility between XML reports in Junit 3 and 4. Will report back here once I've worked out what's going on, but in the meantime can we assume Junit 4 compatiblity in sbt (and, presumably, downstream tools?)
  • an alternative strategy would be to delegate to the specific test library to handle this. Specs2 and scalatest already both have support for outputting Junit XML so it would remove some duplication - although I might need some help getting this to work correctly, and we'd need to decide what to do if a different (non-specs/non-scalatest) framework is being used
@eed3si9n

This comment has been minimized.

Show comment
Hide comment
@eed3si9n

eed3si9n Nov 29, 2016

Member

Generally our preference for new feature is sbt 1.0 especially if it breaks binary compatibility, but give that it's not out yet, if you want to use the feature sooner you'd have to send us a PR first to 0.13, and then forward port it to 1.0.

Member

eed3si9n commented Nov 29, 2016

Generally our preference for new feature is sbt 1.0 especially if it breaks binary compatibility, but give that it's not out yet, if you want to use the feature sooner you'd have to send us a PR first to 0.13, and then forward port it to 1.0.

ashleymercer pushed a commit to ashleymercer/sbt that referenced this issue Nov 29, 2016

Ashley Mercer
JUnitXmlTestsListener now correctly flags ignored, skipped and pendin…
…g tests


Fixes #2198 Ref #PULL

Generated Junit-style XML reports now include a count of ignored,
skipped and pending tests; and individual tests are correctly flagged
with the <skipped/> element.
@ashleymercer

This comment has been minimized.

Show comment
Hide comment
@ashleymercer

ashleymercer Nov 29, 2016

Pull request against 0.13 above, the change is very small so it should be easy for me to forward-port to 1.0

  • junit reports don't distinguish between ignored, skipped and pending tests, so we treat all three as "skipped" in the XML report
  • I've opted for the minimal change for now - modifying the existing listener, rather than trying to forward to the correct test library if it's present - to avoid breaking anything

Pull request against 0.13 above, the change is very small so it should be easy for me to forward-port to 1.0

  • junit reports don't distinguish between ignored, skipped and pending tests, so we treat all three as "skipped" in the XML report
  • I've opted for the minimal change for now - modifying the existing listener, rather than trying to forward to the correct test library if it's present - to avoid breaking anything

ashleymercer pushed a commit to ashleymercer/sbt that referenced this issue Dec 1, 2016

Ashley Mercer
JUnitXmlTestsListener now correctly flags ignored, skipped and pendin…
…g tests


Fixes #2198 Ref #2854

Generated Junit-style XML reports now include a count of ignored,
skipped and pending tests; and individual tests are correctly flagged
with the <skipped/> element.

@eed3si9n eed3si9n closed this in #2858 Dec 2, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment