Change Threshold direction

[aremiuq]

Hello!

I find precrec a great package for calculate ROC/PRC. Nevertheless, I need change the direction of the threshold evaluation. In other words, I want to compare methods than the positive values need to be higher than the threshold, and others than need to be lower than the threshold (for example a fisher's exact test). How can I do it using Precrec?

[takayasaito]

Hi,

I'm not sure if I understand your comment correctly or not, but you can explicitly specify the positive label with posclass from the mmdata function. You can also specify it in the evalmod function that passes unused parameters to mmdata internally. The example below shows two cases: sscurves with positive label of 1 (default) and sscurves2 with positive label of -1 (with posclass).

> library(precrec)
> data(P10N10)
> P10N10$labels
 [1]  1  1 -1  1  1  1  1  1 -1 -1 -1 -1  1 -1  1 -1  1 -1 -1 -1
 
> sscurves <- evalmod(scores = P10N10$scores, labels = P10N10$labels)
> sscurves

    === AUCs ===

     Model name Dataset ID Curve type       AUC
   1         m1          1        ROC 0.7200000
   2         m1          1        PRC 0.7397716


    === Input data ===

     Model name Dataset ID # of negatives # of positives
   1         m1          1             10             10

> sscurves2 <- evalmod(scores = P10N10$scores, labels = P10N10$labels, posclass=-1)
> sscurves2

    === AUCs ===

     Model name Dataset ID Curve type       AUC
   1         m1          1        ROC 0.2800000
   2         m1          1        PRC 0.3695701


    === Input data ===

     Model name Dataset ID # of negatives # of positives
   1         m1          1             10             10