Analyse results routine #202
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds some code I wrote a while ago to generate a ROC curve (graph of true positive rate versus false positive rate) by optimally combining the results of different tests together to give the best possible TPR for any FPR. I've adapted it to the new sqlite database infrastructure and I've moved some of the database routines into a new dbutils.py file as they are used by both summarize-results and the new analyse-results programs.
The analyse-results program is fairly simple. It does an AND on the different QC tests to make combinations and calculates the TPR and FPR of each. The test combination that has the highest ratio of TPR increase to FPR increase is selected to make the next point in the ROC curve, and this continues until there are no more test combinations that give a useful increase in TPR. The program seems to work fairly well given the simple approach, but memory requirements will start to become an issue if the number of QC checks expand much.
I've also included functionality that checks the CSIRO_wire_break results and only uses profile levels above an XBT wire break, in order to allow us to concentrate on other error modes.