allow testing meta_estimators from outside scikit-learn

[niedakh]

Hi,

I'm working on scikit-multilearn and I was wondering if you could help me out a little so that I can use scikit's check_estimator on multi-label classifiers that are not in scikit's repo.

The code adds a possibility to treat an external classifier as what scikit-learn would call a meta estimator.

[amueller]

We should fix this, but maybe not the way you did.
The hard-coded constants are certainly bad (and there are several of them).

I think a meta-estimator should identify itself (by the mixin?), so that the tests know.
We currently have estimator_type but that was likely a bad decision. Maybe that should be a set or a dict? Currently it's a string, and meta-estimator have the estimator type of the thing they wrap (in the case of pipelines at least - your classifier would always be a classifier).

[niedakh]

So a MetaEstimatorMixin inheriting from a ClassifierMixin that adds a function is_metaestimator() {return True} is a good choice?

[amueller]

Sorry for the silence. The solution is this: #6599
So yes, probably a base-class and add estimator_type='meta-estimator' or something like that.
Unfortunately, that will not really run any tests, only the very weak one we currently run. We should probably enable all test to be run by meta-estimators by using a factory.

[amueller]

You can now pass instances: #9019 that should solve your problem. #8022 should further help by allowing instantiation of meta-estimator classes within the common tests. Any feedback welcome!