[core] Fix flaky SummaryHTMLRenderer #4750

219sansim · 2023-11-23T05:29:34Z

Describe the PR

The test SummaryHTMLRendererTest.testRendererMultiple assumes a particular order for the keys stored in summary map.
The following lines of code in renderSummary function may return any random field ordering. This might cause inconsistent output rendering and results in the unit test being flaky.

pmd/pmd-core/src/main/java/net/sourceforge/pmd/renderers/SummaryHTMLRenderer.java

Lines 64 to 72 in bdc08e4

    
           Map<String, MutableInt> summary = getSummary(report); 
        
           for (Entry<String, MutableInt> entry : summary.entrySet()) { 
        
               String ruleName = entry.getKey(); 
        
               writer.write("<tr><td>"); 
        
               writer.write(ruleName); 
        
               writer.write("</td><td align=center>"); 
        
               writer.write(String.valueOf(entry.getValue().intValue())); 
        
               writer.println("</td></tr>"); 
        
           }

Changing the HashMap to LinkedHashMap, makes the output consistent and fixes the flaky test. There might be other possible fixes like, changing the test code such that, the tests expects and passes for both the orderings.

Steps to reproduce the flaky test

Setup the NonDex tool in your repo and run

mvn -pl pmd-core -Dtest=net.sourceforge.pmd.renderers.SummaryHTMLRendererTest#testRendererMultiple

Error Message

net.sourceforge.pmd.renderers.SummaryHTMLRendererTest.testRendererMultiple  Time elapsed: 0.002 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: 
expected: <<html><head><title>PMD</title></head><body>
<center><h2>Summary</h2></center>
<table align="center" cellspacing="0" cellpadding="3">
<tr><th>Rule name</th><th>Number of violations</th></tr>
<tr><td>Boo</td><td align=center>1</td></tr>
<tr><td>Foo</td><td align=center>1</td></tr>
</table>
<center><h2>Detail</h2></center><table align="center" cellspacing="0" cellpadding="3"><tr>
<center><h3>PMD report</h3></center><center><h3>Problems found</h3></center><table align="center" cellspacing="0" cellpadding="3"><tr>
<th>#</th><th>File</th><th>Line</th><th>Problem</th></tr>
<tr bgcolor="lightgrey"> 
<td align="center">1</td>
<td width="*%"><a href="link_prefixnotAvailable.html#line_prefix1">notAvailable</a></td>
<td align="center" width="5%">1</td>
<td width="*">blah</td>
</tr>
<tr> 
<td align="center">2</td>
<td width="*%"><a href="link_prefixnotAvailable.html#line_prefix1">notAvailable</a></td>
<td align="center" width="5%">1</td>
<td width="*">blah</td>
</tr>
</table></tr></table></body></html>
> but was: <<html><head><title>PMD</title></head><body>
<center><h2>Summary</h2></center>
<table align="center" cellspacing="0" cellpadding="3">
<tr><th>Rule name</th><th>Number of violations</th></tr>
<tr><td>Foo</td><td align=center>1</td></tr>
<tr><td>Boo</td><td align=center>1</td></tr>
</table>
<center><h2>Detail</h2></center><table align="center" cellspacing="0" cellpadding="3"><tr>
<center><h3>PMD report</h3></center><center><h3>Problems found</h3></center><table align="center" cellspacing="0" cellpadding="3"><tr>
<th>#</th><th>File</th><th>Line</th><th>Problem</th></tr>
<tr bgcolor="lightgrey"> 
<td align="center">1</td>
<td width="*%"><a href="link_prefixnotAvailable.html#line_prefix1">notAvailable</a></td>
<td align="center" width="5%">1</td>
<td width="*">blah</td>
</tr>
<tr> 
<td align="center">2</td>
<td width="*%"><a href="link_prefixnotAvailable.html#line_prefix1">notAvailable</a></td>
<td align="center" width="5%">1</td>
<td width="*">blah</td>
</tr>
</table></tr></table></body></html>
>

Ready?

Added unit tests for fixed bug/feature
Passing all unit tests
Complete build ./mvnw clean verify passes (checked automatically by github actions)
Added (in-code) documentation (if needed)

pmd-test · 2023-11-23T17:02:22Z

	1 Warning
⚠️	Running pmdtester failed, this message is mainly used to remind the maintainers of PMD.

Generated by 🚫 Danger

adangel

LGTM, thanks!

[core] Fix flaky SummaryHTMLRenderer #4750

Fix flaky SummaryHTMLRenderer

49bf440

adangel changed the title ~~Fix flaky SummaryHTMLRenderer~~ [core] Fix flaky SummaryHTMLRenderer Nov 23, 2023

adangel added the a:bug PMD crashes or fails to analyse a file. label Nov 23, 2023

adangel added this to the 7.0.0 milestone Nov 23, 2023

adangel approved these changes Nov 23, 2023

View reviewed changes

adangel added a commit that referenced this pull request Nov 23, 2023

[doc] Update release notes (#4750)

a8e80cc

adangel added a commit that referenced this pull request Nov 23, 2023

Merge pull requets #4750 from 219sansim:fix_flaky_SummaryHTMLRenderer

f030d7f

[core] Fix flaky SummaryHTMLRenderer #4750

adangel merged commit 49bf440 into pmd:master Nov 23, 2023
3 checks passed

219sansim mentioned this pull request Dec 3, 2023

Add PMD fixed tests TestingResearchIllinois/idoft#1159

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[core] Fix flaky SummaryHTMLRenderer #4750

[core] Fix flaky SummaryHTMLRenderer #4750

219sansim commented Nov 23, 2023

pmd-test commented Nov 23, 2023

adangel left a comment

	Map<String, MutableInt> summary = getSummary(report);
	for (Entry<String, MutableInt> entry : summary.entrySet()) {
	String ruleName = entry.getKey();
	writer.write("<tr><td>");
	writer.write(ruleName);
	writer.write("</td><td align=center>");
	writer.write(String.valueOf(entry.getValue().intValue()));
	writer.println("</td></tr>");
	}

[core] Fix flaky SummaryHTMLRenderer #4750

[core] Fix flaky SummaryHTMLRenderer #4750

Conversation

219sansim commented Nov 23, 2023

Describe the PR

Steps to reproduce the flaky test

Ready?

pmd-test commented Nov 23, 2023

adangel left a comment

Choose a reason for hiding this comment