Implement "Reject Option Classification" post-processing technique

[cosmicBboy]

Single Classifier Setting

• training an initial classifier on dataset D
• generating predicted probabilities on the test set
• computing the proximity of each prediction to the decision boundary learned by the
  classifier
• within the critical region threshold theta around the decision boundary,
  where 0.5 < theta < 1, X_s1 (disadvantaged observations) are assigned as y+ and
  X_s0 (advantaged observations are assigned as y –.

Multi-classifier Setting

ROC in the multiple classifier setting is similar to the single classifier setting, except that predicted probabilities are defined as the weighted average of probabilities generated by each classifier C_k (k is the number of different classifiers trained), where the weights can be defined as:

• the accuracy of the classifier on the data.
• uniform (take the mean of the predictions)