Add multi-label support to the confusion matrix metric

[arjoly]

Currently the confusion_matrix support binary and multi-class classification, but not multi-label data yet.

[jnothman]

What does confusion matrix mean in a multilabel context?

On 20 July 2014 20:34, Arnaud Joly notifications@github.com wrote:

Currently the confusion_matrix support binary and multi-class
classification, but not multi-label data yet.

—
Reply to this email directly or view it on GitHub
#3452.

[arjoly]

The same as in the multiclass case, but the notion of true positive, true negative, false negative are different the multi-label case.

For example, this is explained in http://www.cnts.ua.ac.be/~vincent/pdf/microaverage.pdf

[jnothman]

Okay, not actually a confusion matrix in the multiclass sense (which label
is confused with which), but a binarized error analysis.

On 20 July 2014 22:50, Arnaud Joly notifications@github.com wrote:

The same as in the multiclass case, but the notion of true positive,
true negative, false negative are different the multi-label case.

For example, this is explain
http://www.cnts.ua.ac.be/~vincent/pdf/microaverage.pdf

—
Reply to this email directly or view it on GitHub
#3452 (comment)
.

[arjoly]

Okay, not actually a confusion matrix in the multiclass sense (which label
is confused with which), but a binarized error analysis.

Then we might want to add a function. It might be binarized_confusion_matrix or multilabel_confusion_matrix.

[Magellanea]

@arjoly @jnothman This implements the multilabel_confusion_matrix function which computes the values True positives, true negatives, false positives and false negatives for a multi-label classification problem, was that what you expected to be the behavior of the multilabel_confusion_matrix

[Florents-Tselai]

What's the status on this?

[arjoly]

The pr #3452 has stalled feel free to continue if you want to finish it.

[pramitchoudhary]

Is there any plan, or is someone working on pr #3452 ?

[Spandan-Madan]

I am planning to add multi label classification metrics, including confusion matrix. There is a NIPS 2012 paper which explains great metrics for multi-label context. Will put in the effort if I think there's enough people who still need it. Is it still needed? Or is to not that important for the community?

[ChristianSch]

Which paper exactly are you referring to? I find the metrics quite ambiguous in general use. Some use F_1/precision/recall/subset accuracy micro/macro and some use other metrics. (I use the former)

I'm always interested in bringing more native ml support to sklearn. But also be aware of other approaches like scikit-multilearn, or my own small skml for scikit compatible multi label classification. I find the things for ml in sklearn lacking and quite unusable.

[arjoly]

Adding the most important multi-label algorithms is very interesting in my opinion. This is indeed a very long term project.

[carlthome]

@Spandan-Madan, which NIPS 2012 paper are you thinking of?