Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix container failure XML reporting #2542

Merged
merged 3 commits into from Feb 1, 2021

Conversation

@marcphilipp
Copy link
Member

@marcphilipp marcphilipp commented Jan 31, 2021

Please see individual commits.

Fixes #2537.

marcphilipp added 2 commits Jan 31, 2021
Prior to this commit, failing containers were only reported in case they
contained at least one test. However, for example for a parameterized
Jupiter tests and an exception in a `@BeforeAll` method, that led to
failures being silently swallowed. Now, in addition to tests, leaf
nodes of the test tree are always included in the XML report, even if
they are containers, not tests.

Fixes #2537.
Prior to this commit, failures on the container level that occur after
their children have been executed were not reported, e.g. when an
exception was thrown from a Jupiter `@AfterAll` method. Now such
failures cause the contained tests to be reported as failed.
@marcphilipp marcphilipp self-assigned this Jan 31, 2021
@codecov
Copy link

@codecov codecov bot commented Jan 31, 2021

Codecov Report

Merging #2542 (582af11) into main (eab6490) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #2542      +/-   ##
============================================
+ Coverage     90.54%   90.57%   +0.03%     
- Complexity     4713     4721       +8     
============================================
  Files           413      413              
  Lines         11653    11665      +12     
  Branches        924      921       -3     
============================================
+ Hits          10551    10566      +15     
  Misses          825      825              
+ Partials        277      274       -3     
Impacted Files Coverage Δ Complexity Δ
...t/platform/reporting/legacy/xml/XmlReportData.java 100.00% <100.00%> (ø) 22.00 <7.00> (-1.00)
...platform/reporting/legacy/xml/XmlReportWriter.java 97.80% <100.00%> (+1.27%) 63.00 <15.00> (+7.00)
...ain/java/org/junit/jupiter/api/AssertionUtils.java 77.41% <0.00%> (-2.59%) 48.00% <0.00%> (-1.00%)
...va/org/junit/vintage/engine/execution/TestRun.java 94.23% <0.00%> (+0.96%) 30.00% <0.00%> (+1.00%)
...t/vintage/engine/execution/RunListenerAdapter.java 93.87% <0.00%> (+2.04%) 45.00% <0.00%> (+2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eab6490...582af11. Read the comment docs.

@marcphilipp marcphilipp merged commit 3583c87 into main Feb 1, 2021
14 checks passed
14 checks passed
@github-actions
Linux
Details
@github-actions
Analyze (java)
Details
@github-actions
OpenJDK 16
Details
@github-actions
Check build reproducibility
Details
@github-actions
Validation
Details
@github-actions
Validation
Details
@github-actions
Analyze (javascript)
Details
@github-actions
OpenJDK 17
Details
@github-actions
Windows
Details
@github-actions
Mac OS
Details
@github-actions
Publish Snapshot Artifacts
Details
@github-actions
Update Snapshot Documentation
Details
@github-code-scanning
CodeQL No new or fixed alerts
Details
@codecov
codecov/project 90.57% (+0.03%) compared to eab6490
Details
@marcphilipp marcphilipp deleted the marc/2537-always-include-failures-in-xml-report branch Feb 1, 2021
marcphilipp added a commit that referenced this pull request Feb 1, 2021
Prior to this commit, failing containers were only reported in case they
contained at least one test. However, for example for a parameterized
Jupiter tests and an exception in a `@BeforeAll` method, that led to
failures being silently swallowed. Now, in addition to tests, leaf
nodes of the test tree are always included in the XML report, even if
they are containers, not tests.

Moreover, failures on the container level that occurred after their
children had been executed were not reported, e.g. when an exception
was thrown from a Jupiter `@AfterAll` method. Now such failures cause
the contained tests to be reported as failed.

Fixes #2537.
mpkorstanje added a commit to mpkorstanje/junit5 that referenced this pull request Feb 6, 2021
Prior to this commit, failing containers were only reported in case they
contained at least one test. However, for example for a parameterized
Jupiter tests and an exception in a `@BeforeAll` method, that led to
failures being silently swallowed. Now, in addition to tests, leaf
nodes of the test tree are always included in the XML report, even if
they are containers, not tests.

Moreover, failures on the container level that occurred after their
children had been executed were not reported, e.g. when an exception
was thrown from a Jupiter `@AfterAll` method. Now such failures cause
the contained tests to be reported as failed.

Fixes junit-team#2537.
This was referenced Mar 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

1 participant