-
-
Notifications
You must be signed in to change notification settings - Fork 223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a to_junit method to standardize the report format of TestSuiteResult #1703
Add a to_junit method to standardize the report format of TestSuiteResult #1703
Conversation
@luca-martial Would you please review my PR ? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR,
my concern is that we lose too much information about the tests when exporting and reading the report don't allow to understand the results.
In my opinion we should generate report like this:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites time="8.605871" tests="3">
<testsuite name="Test accuracy (model=..., dataset=..., metric=0.5)" time="10.605871">
<testcase name="metric_name" time="6.605871" />
</testsuite>
<testsuite name="Test F1 (model=..., dataset=..., metric=0.5)" time="2">
<testcase name="metric_name" time="2">
<failure message="Test failed with metric of 0.3" type="TestFailed">
<!-- Full Test result information here (metric, output_ds, messages, ...) -->
</failure>
</testcase>
</testsuite>
<testsuite name="Test with error (model=..., dataset=..., metric=0.5)" time="2">
<testcase name="metric_name" time="2">
<failure message="Insert error message here" type="Error">
<!-- stacktrace given in result.message) -->
</failure>
</testcase>
</testsuite>
</testsuites>
…b.com/Kranium2002/giskard into improvement/standardize-report-format
@kevinmessiaen I still need help in implementing the runtime calculation logic. Would you please help me with that? |
Sure, let me look at it |
Let's merge without the execution time of each tests at first. I created an issue to tackle the problem of adding execution time: #1759 |
Description
This PR adds to_Junit function to TestSuitResult Class
Related Issue
This PR is related to issue number #1685
Type of Change
Checklist
CODE_OF_CONDUCT.md
document.CONTRIBUTING.md
guide.make codestyle
.