DM-32819: 'verifyFlatStatistics' returns "RXX_S00 SUCCESS" when listing failures #16
Conversation
| if testResult is False: | ||
| calcStats['FAILURES'].append(testName) | ||
| detSuccess = detStats['VERIFY']['DET'].pop('SUCCESS', False) | ||
| if not detSuccess: |
There was a problem hiding this comment.
It seems like any dict with a truthy SUCCESS value will pass here, including {'SUCCESS': [], 'FAIL': False}. I gather this is meant to short-circuit checking if all tests passed, but it seems unnecessarily error-prone if it's just for optimization. Is there any other purpose to the SUCCESS value?
There was a problem hiding this comment.
Good question. I believe the intent is to keep all the logic of what constitutes a failure in the subclasses and to have this merge code just handle collating the results. I'm pinging @czwa ;)
There was a problem hiding this comment.
There's this ticket too: https://jira.lsstcorp.org/browse/DM-29643
There was a problem hiding this comment.
Yeah fair enough, if there's an underlying issue it should be fixed on that ticket, but I'd still like to know if the existing design is working as intended before this PR is merged.
There was a problem hiding this comment.
The goal of SUCCESS is to save the merge code from having to decide if it should create/append to a list of failures. Some optional tests are excluded from that check, and are calculated to help debugging larger issues.
No description provided.