Skip to content

Conversation

fredjn
Copy link
Member

@fredjn fredjn commented Aug 13, 2024

Description of the Change

Since we no longer collect information/events on subsuites
the logic fo logging messages about failures in sub suites
was not workin properly. This resulted in incorrect error
message when a suite successfully executed but had test case
failures in it.

Possible Drawbacks

Log message will be a tad less detailed in it's information (e.g no information about the number of sub suites)

Sign-off

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Fredrik Fristedt <fredrik.fristedt@axis.com>

Since we no longer collect information/events on subsuites
the logic fo logging messages about failures in sub suites
was not workin properly. This resulted in incorrect error
message when a suite successfully executed but had test case
failures in it.
@fredjn fredjn requested a review from a team as a code owner August 13, 2024 06:26
@fredjn fredjn requested review from t-persson and andmat900 and removed request for a team August 13, 2024 06:26
if sub_suites == 0:
return False, "ETOS failed to start any test suites"
return False, f"{failures}/{sub_suites} test suites failed."
return False, f"Test case failures during test suite execution"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to double-check, ignore if it is out-of-scope:

If the suite JSON file contains an empty list, will this be handled by this method? I remember I saw weird error messages in that case before.

Copy link
Member Author

@fredjn fredjn Aug 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a check for empty suite definition list (see eiffel-community/etos-api#71) and I also checked some other sad path cases with broken suite definitions. It seems to work as expected.

@fredjn fredjn merged commit c1f5ed5 into eiffel-community:main Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants