Extend ClassifierChain to multi-output problems

[jnothman]

ClassifierChain currently supports multilabel classification. It should be straightforward to extend it to multi-output (as long as it only chains on predict) except for implementing ClassifierChain.{predict_proba,decision_function} which will take some care.

[siebenHeaven]

@jnothman I am new to contributing this project. Would like to start here. The way i understand this is that currently, ClassifierChain predicts that a given instance belongs to a class, and goes on (by passing this prediction to next estimator in the chain) to check for other classes.So would extending it to support multi-output include implementing a new parameter which, if set, tells it to not check for further classes?

[jnothman]

no it's basically the same structure. at the moment it is multilabel, which is equivalent to multi-output for binary problems. The chain consists of a collection of binary classifiers. Extending to multi-output multiclass (which I should have stated more explicitly above) just means each classifier may be multiclass.

…

On 3 Jul 2017 12:39 am, "siebenHeaven" ***@***.***> wrote: @jnothman <https://github.com/jnothman> I am new to contributing this project. Would like to start here. The way i understand this is that currently, ClassifierChain predicts that a given instance belongs to a class, and goes on (by passing this prediction to next estimator in the chain) to check for other classes.So would extending it to support multi-output include implementing a new parameter which, if set, tells it to not check for further classes? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9245 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz67V5r4kXdHBwDYJlAd4LIajkWGB8ks5sJ6ujgaJpZM4OJHOm> .

[siebenHeaven]

Ok! So right now, the base estimator that is passed to the ClassifierChain is a binary classifier. So inorder to make multi-class classifiers work, what changes will be needed? Would it need a separate class or changes would be needed in the current ClassifierChain class?
Also, what dataset can this multi-outptut multiclass be tested on?
(sorry for late reply,had some work at university :) )

[jnothman]

No, use the same class. Basically, you just need to ensure that the predict_proba and decision_function output conform to what you get from a multi-output DecisionTreeClassifier. It doesn't look like we have any standard datasets here. You could take a look at sklearn/tree/tests/test_tree.py:test_multioutput.

[Johayon]

I could take it up, if no one is currently working on it.

[jnothman]

you may, @Johayon.

[agamemnonc]

@jnothman just to clarify, by multi-output you mean multi-class, right? Because I feel that the convention that is currently used is that multi-output == multi-label.

Anyway, I confirm that this is currently causing issues, especially given that the outputs of MultioutputClassifier and ClassifierChain predict_proba are not compatible; the former returns a list of length n_outputs where each element has shape = (n_samples, n_classes), whereas the latter returns an array of shape = (n_samples, n_outputs).

For multi-output binary problems, this is OK, as it is assumed that the method returns the probability of the positive class for each output. However, in the multioutput-multiclass case, what predict_proba returns makes no sense, since there is no way to know which class the probabilities correspond to. And there are no warning messages to let the user know that the results may not make sense. Therefore, I would personally classify this issue as a bug.

@Johayon if you have decided not to work on this any more, pls let me know and I would be happy to take it up.

[jnothman]

Yes, I mean multioutput multiclass. And we know that there are inconsistencies in predict_proba for multilabel, although that's better described with respect to decision_function: http://scikit-learn.org/0.20/glossary.html#term-decision-function Yes, I think we should ideally be working towards a more consistent probability representation for multilabel, beginning with deprecating OneVsRestClassifier for multilabel... But other core devs may disagree

[henrif94]

I am still getting a No Loop Matching Error for .predict_proba().
Any Updates on this issue?

[agamemnonc]

There is a PR under-way to address this issue (#14654).

It is currently in stall mode due to conflicts with the master etc, but I hope to be able to address this in the next few weeks.

[lucyleeow]

Looking deeper into this, Y is currently of shape (n_samples, n_classes) (multi-label binarized). Other estimators that support multi-label, multi-output have Y of shape (n_samples, n_outputs). I could not find an example where Y is of shape list of (n_samples, n_classes).

If we want to implement this, do we ask Y to be:

1. (n_samples, n_outputs) (not backwards compatible)
2. List of (n_samples, n_classes) - ? complex?
3. Support both to be backwards compatible - difficult to maintain?

cc @ogrisel since you reviewed #21942