Skip to content

Conversation

@vkarak
Copy link
Contributor

@vkarak vkarak commented Feb 24, 2021

Fixes #1812.

@vkarak vkarak added this to the ReFrame 3.5.0 milestone Feb 24, 2021
@vkarak vkarak requested a review from ekouts February 24, 2021 17:57
@vkarak vkarak self-assigned this Feb 24, 2021
@codecov-io
Copy link

Codecov Report

Merging #1814 (79e7a1e) into master (0c929f4) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1814   +/-   ##
=======================================
  Coverage   87.60%   87.60%           
=======================================
  Files          49       49           
  Lines        7937     7937           
=======================================
  Hits         6953     6953           
  Misses        984      984           
Impacted Files Coverage Δ
reframe/frontend/executors/__init__.py 98.19% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c929f4...79e7a1e. Read the comment docs.

Copy link
Contributor

@ekouts ekouts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks good to me but some lines below there is another bug from my maxfail PR:

            num_completed = len(self._stats.completed())
...
            self._printer.status(
                status,
                status,
                f'Ran {num_completed}/{len(testcases)}'
                f' test case(s) from {num_checks} check(s) '
                f'({num_failures} failure(s))',
                just='center'
            )

In this line f'Ran {num_completed}/{len(testcases)}', num_completed will be only the ones that were completed in the last retry. Not sure how we to fix this simply, since completed are the ones that failed or succeeded and we cannot just sum the completed from all runs.

(edit)
Now that I think about it the number we want to print there is len(testcases) - len(self._stats.tasks()) + len(self._stats.completed()).

@vkarak
Copy link
Contributor Author

vkarak commented Mar 1, 2021

@jenkins-cscs retry daint dom

@vkarak vkarak merged commit 48135c7 into reframe-hpc:master Mar 1, 2021
@vkarak vkarak deleted the bugfix/retries-failure-result branch March 1, 2021 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ReFrame erroneously reports FAILURE when tests pass in retries

3 participants