FMI Standard Tool listing shows "passed" despite "notCompliantWithLatestTestRules" file

[ghorwin]

For example:

results/2.0/cs/linux64/SIMPACK/9.10.1/JModelica.org/1.15/ControlledTemperature/

Directory contains both passed and notCompliantWithLatestRules files. Fmi-Tools-Listing shows corresponding case still as validated (btw, results are way off, so 'failed' would have been the appropriate rating).

Currently, the webpage display does not allow to distinguish between "had been validated with previous set of cross-check-rules" and "does comply to current cross-check-rules".

Suggest: add special handling in validate_vendor_repo script.

Problem appears to be in lines 277-278:validate_vendor_repo.py where

        if new_problems and clean_up:
            not_compliant_file = os.path.join(subdir, 'notCompliantWithLatestRules')

notCompliant... file is only tested for, when running with cleanup and when problems were found.
Maybe the check for 'notComplient' could be moved ahead and the test case should be skipped alltogether if the file is present?

(Note to self: I guess I have to read up on the discussion on that issue in the meeting notes... :-) )

[lochel]

This issue is concerning. Can please someone comment on it? Does the described issue actually falsify the displayed results?

[lochel]

@ghorwin I think this is finally fixed and can be closed. See #129 (comment)