Add support for JUnit reports

[onlineque]

fixes #10

[kehoecj]

@onlineque Let me know when this is ready for review

[onlineque]

@kehoecj It should be ready for the review, could you please take a look ?

[kehoecj]

@onlineque Test coverage has dropped below the required coverage threshold (95%) which is causing the unit tests job to fail. Please improve the coverage so that is meets the threshold.

[jd4235]

When I build and run locally to generate a junit output, I see the following entry that appears to use HTML entity codes in the XML. Why are those codes being used? They are not used in the other reporter output formats (json and standard)

     <testcase name="fixtures/subdir/bad.yml validation" classname="config-file-validator" file="fixtures/subdir/bad.yml">
       <failure>
         <message>yaml: line 3: did not find expected &#39;-&#39; indicator</message>
       </failure>
     </testcase>

[jd4235]

Please increase your test coverage so that the coverage is above 95% and the pipeline succeeds.

[onlineque]

When I build and run locally to generate a junit output, I see the following entry that appears to use HTML entity codes in the XML. Why are those codes being used? They are not used in the other reporter output formats (json and standard)

     <testcase name="fixtures/subdir/bad.yml validation" classname="config-file-validator" file="fixtures/subdir/bad.yml">
       <failure>
         <message>yaml: line 3: did not find expected &#39;-&#39; indicator</message>
       </failure>
     </testcase>

it renders correctly once opened in any browser / tool which is able to render XML. From my point of view this is correct.

[kehoecj]

When I build and run locally to generate a junit output, I see the following entry that appears to use HTML entity codes in the XML. Why are those codes being used? They are not used in the other reporter output formats (json and standard)

     <testcase name="fixtures/subdir/bad.yml validation" classname="config-file-validator" file="fixtures/subdir/bad.yml">
       <failure>
         <message>yaml: line 3: did not find expected &#39;-&#39; indicator</message>
       </failure>
     </testcase>

it renders correctly once opened in any browser / tool which is able to render XML. From my point of view this is correct.

It really only looks good in the browser. Viewing in VSCode, on the command line, or any other tool shows the HTML encoded output which is difficult to read. The primary intent for offering a Junit reporter - at least in my mind - was for use with CI tools like Gitlab CI that can take a JUnit XML as a report.

[kehoecj]

Code looks good overall with a few comments. I tested locally and was able to generate valid JUnit XML. Tested the output in a sample Gitlab CI job using the JUnit reporter and which worked as expected to include config-file-validation reporting in the Gitlab CI pipeline testing section.

A couple code changes/questions and the remaining question on encoded XML

[onlineque]

When I build and run locally to generate a junit output, I see the following entry that appears to use HTML entity codes in the XML. Why are those codes being used? They are not used in the other reporter output formats (json and standard)

     <testcase name="fixtures/subdir/bad.yml validation" classname="config-file-validator" file="fixtures/subdir/bad.yml">
       <failure>
         <message>yaml: line 3: did not find expected &#39;-&#39; indicator</message>
       </failure>
     </testcase>

it renders correctly once opened in any browser / tool which is able to render XML. From my point of view this is correct.

It really only looks good in the browser. Viewing in VSCode, on the command line, or any other tool shows the HTML encoded output which is difficult to read. The primary intent for offering a Junit reporter - at least in my mind - was for use with CI tools like Gitlab CI that can take a JUnit XML as a report.

I don't think a junit report will anyone read from a command line, it's not its intention, or do I miss something ?
It really looks good even in the Gitlab CI, it renders correctly, I made a test:

[kehoecj]

When I build and run locally to generate a junit output, I see the following entry that appears to use HTML entity codes in the XML. Why are those codes being used? They are not used in the other reporter output formats (json and standard)

     <testcase name="fixtures/subdir/bad.yml validation" classname="config-file-validator" file="fixtures/subdir/bad.yml">
       <failure>
         <message>yaml: line 3: did not find expected &#39;-&#39; indicator</message>
       </failure>
     </testcase>

it renders correctly once opened in any browser / tool which is able to render XML. From my point of view this is correct.

It really only looks good in the browser. Viewing in VSCode, on the command line, or any other tool shows the HTML encoded output which is difficult to read. The primary intent for offering a Junit reporter - at least in my mind - was for use with CI tools like Gitlab CI that can take a JUnit XML as a report.

I don't think a junit report will anyone read from a command line, it's not its intention, or do I miss something ? It really looks good even in the Gitlab CI, it renders correctly, I made a test:

if you look at Pytest and its junit output it does not HTML encode the failure message:

<?xml version="1.0" encoding="utf-8"?><testsuites><testsuite name="pytest" errors="0" failures="1" skipped="0" tests="1" time="0.070" timestamp="2023-11-30T14:32:30.709146" hostname="A6354835"><testcase classname="test" name="test_junitoutput_special_chars" time="0.001"><failure message="Exception: yaml: line 3: did not find expected '-' indicator">def test_junitoutput_special_chars():
&gt;       raise Exception("yaml: line 3: did not find expected '-' indicator")
E       Exception: yaml: line 3: did not find expected '-' indicator

test.py:2: Exception</failure></testcase></testsuite></testsuites>

It also renders just fine on HTML:

So it seems like it's possible to get HTML-based and non-html-based outputs to look the same which is what we want if possible

[jd4235]

Nice job. Thanks for fixing the XML output to be human readable everywhere.

[kehoecj]

LGTM - thank you for the contribution!