Fast, weighted ROC curves
Switch branches/tags
Nothing to show
Clone or download
Latest commit 6cf7b30 Oct 2, 2018
Permalink
Failed to load latest commit information.
R simple data set after require Oct 1, 2018
man simple data set after require Oct 1, 2018
tests simplify vignette Oct 1, 2018
vignettes simplify vignette Oct 1, 2018
.Rbuildignore simplify vignette Oct 1, 2018
.gitignore build version++ Aug 25, 2017
.travis.yml travis, times=10 Jun 8, 2017
DESCRIPTION pull from master Oct 1, 2018
NAMESPACE passes check Jul 7, 2014
NEWS simplify vignette Oct 1, 2018
README.org PRROC Aug 25, 2017
build.sh build version++ Aug 25, 2017

README.org

Fast, weighted ROC curves

testshttps://travis-ci.org/tdhock/WeightedROC.png?branch=master
coveragehttps://coveralls.io/repos/tdhock/WeightedROC/badge.svg?branch=master&service=github

Receiver Operating Characteristic (ROC) curve analysis is one way to evaluate an algorithm for binary classification. R packages ROCR/pROC/AUC/PerfMeas/PRROC implement ROC curve computation. However, if the observations have weights (non-uniform loss, see Definition vignette) then these packages can not be used. The WeightedROC package implements ROC and Area Under the Curve (AUC) computation for weighted binary classification problems.

Installation

From CRAN:

install.packages("WeightedROC")

From GitHub:

if(!require(devtools))install.packages("devtools")
devtools::install_github("tdhock/WeightedROC")

Usage

library(WeightedROC)
example(WeightedROC)
example(WeightedAUC)

Comparison with other R packages implementing ROC curve computation

Packageversiondatelines of R codeweightstestscumsum
pROC1.7.92014-06-125666nonoyes
ROCR1.0-52013-05-161650nonoyes
PerfMeas1.2.12014-09-07684nonono
PRROC1.32017-04-21610yesyesyes
AUC0.3.02013-09-30354nonono
WeightedROC2017.08.122017-08-12288yesyesyes
glmnet::auc1.9-52013-08-0122yesnoyes
  • weights shows which packages allow weights (non-uniform loss for each observation).
  • tests shows which R packages implement unit tests to check that the ROC/AUC is computed correctly.
  • lines of R code shows how many lines of code were used in the pkg/R/* files. Note that WeightedROC has the simplest implementation other than glmnet::auc.
  • cumsum shows whether or not the cumsum function is used to compute the ROC curve. Using the cumsum function is simple to code and fast – see the Speed vignette.

When to use PRROC?

For “soft” real-valued labels (not “hard” labels $∈ \{-1, 1\}$), and accurate interpolation of Precision-Recall curves, use PRROC. Note that PRROC uses the word “Weighted” to mean something completely different (soft labels) than the weights in this package (non-uniform loss), as explained in their PLOS ONE paper.

When to use ROCR?

To compute other evaluation metrics (e.g. lift) use the ROCR package. WeightedROC does not implement evaluation metrics other than ROC/AUC.

When to use pROC?

To compute the partial AUC and compare curves using statistical tests use the pROC package. WeightedROC does not implement these features.

When to use glmnet?

The glmnet package includes an auc function for computing AUC, but does not include a function for computing the ROC curve. So it actually can compute the AUC faster than WeightedROC, for both equal or unequal weights. WARNINGS:

  • make sure the class labels are either 0 or 1 (not factors, not -1 or 1 – these will give the incorrect result, with no warning/error).
  • if the data set has tied scores AND weights, glmnet::auc computes something different, see example(WeightedAUC).