Should action fail when some tests are failing?

[ArtiomTr]

Continuation of #91

The original design of this action implies the following behavior:

• Collecting coverage
• Fail if no report.json found
• If coverage threshold is specified, fail when total coverage is less than specified one

However, by #44 PR, new functionality was implemented: annotating on failed tests. With this check, it wouldn't be logical to complete the action successfully when the test fails.

This leads to irrational behavior. I would like to ask, would it be better if the action fails when the tests are failing? Or it should work as now?

Thank you in advance.

[mrchief]

I would like to ask, would it be better if the action fails when the tests are failing? Or it should work as now?

My 2 cents:

• The overall status of tests should be a failure (exit code 1)
• Status of Annotations should not be a failure unless the annotations API call itself fails and thus, not able to create annotations.

[ArtiomTr]

I think this is the most reasonable solution. Thank you!

[ArtiomTr]

For everyone who is interested in the question - the discussion is not closed, if you do not agree with the decision, feel free to express your opinion!

Task status could be tracked here #98