Skip to content

[JENKINS-8713] Display skipped test messages in results #889

Closed
Closed
[JENKINS-8713] Display skipped test messages in results#889
Labels
component:junit-pluginenhancementimported-jira-issuepriority:majorresolution:fixed

Description

@jenkins-infra-bot
jenkins-infra-bot
opened on Feb 8, 2011

JUnit report XML can contain a message for skipped tests, which often contains information regarding why the test is being skipped. It would be very useful if this message could be output in the test results in a similar way to how the failure are shown.

I've attached an example JUnit style report with a skipped test.

Originally reported by davehunt, imported from: Display skipped test messages in results
  • assignee: mc1arke
  • status: Resolved
  • priority: Major
  • component(s): junit-plugin
  • resolution: Fixed
  • resolved: 2013-03-13T22:34:00+00:00
  • votes: 10
  • watchers: 14
  • imported: 2025-11-26
Raw content of original issue 
JUnit report XML can contain a message for skipped tests, which often contains information regarding why the test is being skipped. It would be very useful if this message could be output in the test results in a similar way to how the failure are shown.



I've attached an example JUnit style report with a skipped test.
1 attachment

Activity

  1. jenkins-infra-bot commented on Apr 7, 2011

    @jenkins-infra-bot
    jenkins-infra-bot
    on Apr 7, 2011
    Author
    nofinator:
    • Original comment link
    • Raw content of original comment: 
      The issue type is Improvement but I think this might be a bug because this used to work quite well.
      


      Any skipped tests were listed as such in the Test Report, and the number of skipped tests was represented by a nice yellow color in the Test Result Trend chart. Even now the Test Result still has a column for number of Skipped tests, but it is always set to 0 in each row even when there are skipped tests in the JUnit report.
      


      One theory: this may have started happening to our team after the Performance Plugin was installed, which I think has its own JUnit parser. However, we have this problem even in jobs that aren't using the Performance Plugin.
      


      Update: Skipped tests started showing up again in the Test Result Trend Chart for us again. Please disregard my comment, which should have been logged as a separate, new bug report instead.

    The issue type is Improvement but I think this might be a bug because this used to work quite well.

    Any skipped tests were listed as such in the Test Report, and the number of skipped tests was represented by a nice yellow color in the Test Result Trend chart. Even now the Test Result still has a column for number of Skipped tests, but it is always set to 0 in each row even when there are skipped tests in the JUnit report.

    One theory: this may have started happening to our team after the Performance Plugin was installed, which I think has its own JUnit parser. However, we have this problem even in jobs that aren't using the Performance Plugin.

    Update: Skipped tests started showing up again in the Test Result Trend Chart for us again. Please disregard my comment, which should have been logged as a separate, new bug report instead.

  2. jenkins-infra-bot commented on Apr 7, 2011

    @jenkins-infra-bot
    jenkins-infra-bot
    on Apr 7, 2011
    Author
    redsolo:
    • Original comment link
    • Raw content of original comment: 
      Hi gregory, this issue should be marked as an improvement as it describes a feature that does not yet exist. Your problem sounds like a bug in an existing feature, so please create a new bug-report describing the problem and what plugins you are using (and version). Plus if you can, add a sample report that contains skipped tests to make easy to reproduce the issue.

    Hi gregory, this issue should be marked as an improvement as it describes a feature that does not yet exist. Your problem sounds like a bug in an existing feature, so please create a new bug-report describing the problem and what plugins you are using (and version). Plus if you can, add a sample report that contains skipped tests to make easy to reproduce the issue.

  3. jenkins-infra-bot commented on Mar 6, 2012

    @jenkins-infra-bot
    jenkins-infra-bot
    on Mar 6, 2012
    Author
    yfyf:
    • Original comment link
    • Raw content of original comment: 
      Bumping this and would like to clear some confusion: skipped tests are counted and the total number is displayed, I think there is no "bug" here, but the feature improvement would be to actually list/display (or at least have an option to do so) the skipped tests the same way failed tests are being listed under Test Result view.

    Bumping this and would like to clear some confusion: skipped tests are counted and the total number is displayed, I think there is no "bug" here, but the feature improvement would be to actually list/display (or at least have an option to do so) the skipped tests the same way failed tests are being listed under Test Result view.

  4. jenkins-infra-bot commented on Mar 13, 2013

    @jenkins-infra-bot
    jenkins-infra-bot
    on Mar 13, 2013
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Michael Clarke
    Path:
    core/src/main/java/hudson/tasks/junit/CaseResult.java
    core/src/main/resources/hudson/tasks/junit/CaseResult/index.jelly
    http://jenkins-ci.org/commit/jenkins/3b58cd294f8266f2e4dc80356bf172c37769030e
    Log:
    [FIXED JENKINS-8713] Display test skipped messages in test results

  5. jenkins-infra-bot
    closed this as completedon Mar 13, 2013

  6. jenkins-infra-bot commented on Mar 13, 2013

    @jenkins-infra-bot
    jenkins-infra-bot
    on Mar 13, 2013
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Michael Clarke
    Path:
    changelog.html
    http://jenkins-ci.org/commit/jenkins/2aa0293858d50b064128a9a74f4d25d3f5c4a5c8
    Log:
    JENKINS-8713 Add changelog entry

    Compare: jenkinsci/jenkins@fe9f676...2aa0293

    –
    You received this message because you are subscribed to the Google Groups "Jenkins Commits" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-commits+unsubscribe@​googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

  7. jenkins-infra-bot commented on Mar 14, 2013

    @jenkins-infra-bot
    jenkins-infra-bot
    on Mar 14, 2013
    Author
    dogfood:

    Integrated in jenkins_main_trunk #2369
    [FIXED JENKINS-8713] Display test skipped messages in test results (Revision 3b58cd294f8266f2e4dc80356bf172c37769030e)
    JENKINS-8713 Add changelog entry (Revision 2aa0293858d50b064128a9a74f4d25d3f5c4a5c8)

    Result = UNSTABLE
    michael.m.clarke : 3b58cd294f8266f2e4dc80356bf172c37769030e
    Files :

    • core/src/main/java/hudson/tasks/junit/CaseResult.java
    • core/src/main/resources/hudson/tasks/junit/CaseResult/index.jelly

    michael.m.clarke : 2aa0293858d50b064128a9a74f4d25d3f5c4a5c8
    Files :

    • changelog.html

  8. jenkins-infra-bot commented on Feb 12, 2014

    @jenkins-infra-bot
    jenkins-infra-bot
    on Feb 12, 2014
    Author
    sahumada:
    • Original comment link
    • Raw content of original comment: 
      Hi,
      


      I've been trying to show my skipped tests without much success. My XML looks like:
      


      

      <?xml version="1.0" encoding="UTF-8"?>
<testsuite name="tst_XmlPatterns" tests="2" failures="0" errors="0" skipped="1" time="0.000">
   <properties>
      <property name="QtVersion" value="5.2.2"/>
      <property name="QTestVersion" value="5.2.2"/>
   </properties>
   <testcase classname="tst_XmlPatterns" name="initTestCase" time="0.000">
      <skipped>tst_xmlpatterns.cpp:120 :: [] Device &#x9;IP:169.254.0.13&#xA;The command line tool (/usr/lib/qt5/bin/xmlpatterns) could not be run, possibly because Qt was incompletely built or installed. No tests will be run.</skipped>
   </testcase>
   <testcase classname="tst_XmlPatterns" name="cleanupTestCase" time="0.000"/>
   <system-out/>
   <system-err/>
</testsuite>

      

      

      but it doesn't show anything. 
      

      Looking at the source code, I see that you added an attribute called "message", so if I change my XML to 
      

      

      <?xml version="1.0" encoding="UTF-8"?>
<testsuite name="tst_XmlPatterns" tests="2" failures="0" errors="0" skipped="1" time="0.000">
   <properties>
      <property name="QtVersion" value="5.2.2"/>
      <property name="QTestVersion" value="5.2.2"/>
   </properties>
   <testcase classname="tst_XmlPatterns" name="initTestCase" time="0.000">
      <skipped message="tst_xmlpatterns.cpp:120 :: [] Device &#x9;IP:169.254.0.13&#xA;The command line tool (/usr/lib/qt5/bin/xmlpatterns) could not be run, possibly because Qt was incompletely built or installed. No tests will be run."/>
   </testcase>
   <testcase classname="tst_XmlPatterns" name="cleanupTestCase" time="0.000"/>
   <system-out/>
   <system-err/>
</testsuite>

      

      

      then it fails because the XML is not valid according to junit-4.xsd for the xUnit plugin I am using.
      

      The error from the xUnit plugin I get is:
      

      

      13:03:18 [xUnit] [INFO] - Starting to record.
13:03:18 [xUnit] [INFO] - Processing Custom Tool
13:03:18 [xUnit] [INFO] - [Custom Tool] - 29 test report file(s) were found with the pattern 'test-results/tst_*.xml' relative to '/var/lib/jenkins/jobs/qt-5-qtxmlpatterns-dev/workspace/ARCH/ARM' for the testing framework 'Custom Tool'.
13:03:18 [xUnit] [ERROR] - The converted file for the result file '/var/lib/jenkins/jobs/qt-5-qtxmlpatterns-dev/workspace/ARCH/ARM/test-results/tst_xmlpatterns.xml' (during conversion process for the metric 'Custom Tool') is not valid. The report file has been skipped.
13:03:18 [xUnit] [INFO] - Failing BUILD because 'set build failed if errors' option is activated.
13:03:18 [xUnit] [INFO] - There are errors when processing test results.
13:03:18 [xUnit] [INFO] - Skipping tests recording.
13:03:18 [xUnit] [INFO] - Stop build.

      

      

      So, is this change incorrect/incomplete or does junit-4.xsd/xUnit need to be updated ?

    Hi,

    I've been trying to show my skipped tests without much success. My XML looks like:

    <?xml version="1.0" encoding="UTF-8"?>
<testsuite name="tst_XmlPatterns" tests="2" failures="0" errors="0" skipped="1" time="0.000">
   <properties>
      <property name="QtVersion" value="5.2.2"/>
      <property name="QTestVersion" value="5.2.2"/>
   </properties>
   <testcase classname="tst_XmlPatterns" name="initTestCase" time="0.000">
      <skipped>tst_xmlpatterns.cpp:120 :: [] Device &#x9;IP:169.254.0.13&#xA;The command line tool (/usr/lib/qt5/bin/xmlpatterns) could not be run, possibly because Qt was incompletely built or installed. No tests will be run.</skipped>
   </testcase>
   <testcase classname="tst_XmlPatterns" name="cleanupTestCase" time="0.000"/>
   <system-out/>
   <system-err/>
</testsuite>

    but it doesn't show anything.

    Looking at the source code, I see that you added an attribute called "message", so if I change my XML to 

    <?xml version="1.0" encoding="UTF-8"?>
<testsuite name="tst_XmlPatterns" tests="2" failures="0" errors="0" skipped="1" time="0.000">
   <properties>
      <property name="QtVersion" value="5.2.2"/>
      <property name="QTestVersion" value="5.2.2"/>
   </properties>
   <testcase classname="tst_XmlPatterns" name="initTestCase" time="0.000">
      <skipped message="tst_xmlpatterns.cpp:120 :: [] Device &#x9;IP:169.254.0.13&#xA;The command line tool (/usr/lib/qt5/bin/xmlpatterns) could not be run, possibly because Qt was incompletely built or installed. No tests will be run."/>
   </testcase>
   <testcase classname="tst_XmlPatterns" name="cleanupTestCase" time="0.000"/>
   <system-out/>
   <system-err/>
</testsuite>

    then it fails because the XML is not valid according to junit-4.xsd for the xUnit plugin I am using.

    The error from the xUnit plugin I get is:

    13:03:18 [xUnit] [INFO] - Starting to record.
13:03:18 [xUnit] [INFO] - Processing Custom Tool
13:03:18 [xUnit] [INFO] - [Custom Tool] - 29 test report file(s) were found with the pattern 'test-results/tst_*.xml' relative to '/var/lib/jenkins/jobs/qt-5-qtxmlpatterns-dev/workspace/ARCH/ARM' for the testing framework 'Custom Tool'.
13:03:18 [xUnit] [ERROR] - The converted file for the result file '/var/lib/jenkins/jobs/qt-5-qtxmlpatterns-dev/workspace/ARCH/ARM/test-results/tst_xmlpatterns.xml' (during conversion process for the metric 'Custom Tool') is not valid. The report file has been skipped.
13:03:18 [xUnit] [INFO] - Failing BUILD because 'set build failed if errors' option is activated.
13:03:18 [xUnit] [INFO] - There are errors when processing test results.
13:03:18 [xUnit] [INFO] - Skipping tests recording.
13:03:18 [xUnit] [INFO] - Stop build.

    So, is this change incorrect/incomplete or does junit-4.xsd/xUnit need to be updated ?

  9. jenkins-infra-bot
    added
    enhancement
    imported-jira-issue
    component:junit-plugin
    priority:major
    resolution:fixed
    on Nov 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:junit-pluginenhancementimported-jira-issuepriority:majorresolution:fixed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions