Meta-estimator by combining classifiers #2575

Closed
mjanv opened this Issue Nov 5, 2013 · 5 comments

Comments

Projects
None yet
5 participants
@mjanv

mjanv commented Nov 5, 2013

In scikit-learn, some homogeneous meta-classifiers exist to combine classifiers (AdaBoost, Bagging,...) by applying differents subdatasets on differents instances of the same classifier. What about heterogeneous ones - combine differents classifiers on the same data training - ?

For my own purpose, i'm writing some of these. My starting point is the work of Kuncheva [1] and Kittler [2]. I've planned to go further and check other strategies. Does "scikit" will be interested by a pull request with some of these algorithms ?

The syntax will look like FeatureUnion and will (obviously) support GridSearchCV and Pipeline.

combined_classifiers = ModelUnion([("svc", SVC(C=0.1)),("sgd",SGDClassifier(penalty="l2"))])

[1]: Kuncheva, L. I., Bezdek, J. C., & Duin, R. P. (2001). Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognition, 34(2), 299-314.
[2]: Kittler, J., Hatef, M., Duin, R. P., & Matas, J. (1998). On combining classifiers. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 20(3), 226-239.

@jnothman

This comment has been minimized.

Show comment Hide comment
@jnothman

jnothman Feb 27, 2014

Owner

I might look at the references later, but can you briefly describe how these models are combined?

Owner

jnothman commented Feb 27, 2014

I might look at the references later, but can you briefly describe how these models are combined?

@arjoly arjoly added the New Feature label Jul 18, 2014

@jnothman

This comment has been minimized.

Show comment Hide comment
@jnothman

jnothman Jun 6, 2015

Owner

@rasbt, @glouppe: Is this feature request satisfied by VotingClassifier?

Owner

jnothman commented Jun 6, 2015

@rasbt, @glouppe: Is this feature request satisfied by VotingClassifier?

@amueller

This comment has been minimized.

Show comment Hide comment
@amueller

amueller Jun 6, 2015

Owner

I feel it is, as the issue is a bit unspecific. we still might want to do stacking or bayesian model averaging but for that there are different issues, I think.

Owner

amueller commented Jun 6, 2015

I feel it is, as the issue is a bit unspecific. we still might want to do stacking or bayesian model averaging but for that there are different issues, I think.

@rasbt

This comment has been minimized.

Show comment Hide comment
@rasbt

rasbt Jun 6, 2015

Contributor

@amueller I saw this the new "issue" on stacking the other day and I think that would nice to have & easy to implement.

@mjanv You asked for a method to "combine differents classifiers on the same data training", and as mentioned by @jnothman one way would be the VotingClassifier that has been merged recently.

unknown-2

Contributor

rasbt commented Jun 6, 2015

@amueller I saw this the new "issue" on stacking the other day and I think that would nice to have & easy to implement.

@mjanv You asked for a method to "combine differents classifiers on the same data training", and as mentioned by @jnothman one way would be the VotingClassifier that has been merged recently.

unknown-2

@amueller

This comment has been minimized.

Show comment Hide comment
@amueller

amueller Aug 28, 2015

Owner

closing as unspecific and we have voting classifier

Owner

amueller commented Aug 28, 2015

closing as unspecific and we have voting classifier

@amueller amueller closed this Aug 28, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment