Skip to content

Log the correct fail/error message #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 30, 2024
Merged

Log the correct fail/error message #252

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 1 addition & 9 deletions cli/src/etos_client/test_results/test_results.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,22 +57,14 @@ def __test_result(self, test_suites: list[TestSuite]) -> tuple[bool, str]:
"""Build test results based on events retrieved."""
if not self.__has_failed(test_suites):
return True, "Test suite finished successfully."

failures = 0
sub_suites = 0
for test_suite in test_suites:
failures += self.__count_sub_suite_failures(test_suite.sub_suites)
sub_suites += len(test_suite.sub_suites)
messages = self.__fail_messages(test_suites)
if len(messages) == 1:
return False, messages[0]
if messages:
for message in messages[:-1]:
self.logger.error(message)
return False, messages[-1]
if sub_suites == 0:
return False, "ETOS failed to start any test suites"
return False, f"{failures}/{sub_suites} test suites failed."
return False, f"Test case failures during test suite execution"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to double-check, ignore if it is out-of-scope:

If the suite JSON file contains an empty list, will this be handled by this method? I remember I saw weird error messages in that case before.

Copy link
Member Author

@fredjn fredjn Aug 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a check for empty suite definition list (see eiffel-community/etos-api#71) and I also checked some other sad path cases with broken suite definitions. It seems to work as expected.


def get_results(self, events: Events) -> tuple[Optional[bool], Optional[str]]:
"""Get results from an ETOS testrun."""
Expand Down