Also allow tuning of the Stacking parameters

[rasbt]

Currently, only the parameters of the first and meta-level classifiers in the stacking implementations can currently be tuned. E.g., the following works

sclf = StackingCVClassifier(classifiers=[clf1, clf2], 
                            meta_classifier=lr)

params = {'use_probas': [True, False],  # add the use_probas parameter to the search space
          'kneighborsclassifier__n_neighbors': [1, 5],
          'randomforestclassifier__n_estimators': [10, 50],
          'meta-logisticregression__C': [0.1, 10.0]}

grid = GridSearchCV(estimator=sclf, 
                    param_grid=params, 
                    cv=5,
                    refit=True)

but the use_probas param of the StackingCVClassifier cannot be tuned; the following code will produce an error.

sclf = StackingCVClassifier(classifiers=[clf1, clf2], 
                            meta_classifier=lr)

params = {'use_probas': [True, False],  # add the use_probas parameter to the search space
          'kneighborsclassifier__n_neighbors': [1, 5],
          'randomforestclassifier__n_estimators': [10, 50],
          'meta-logisticregression__C': [0.1, 10.0],
          'use_probas': [True, False]}

grid = GridSearchCV(estimator=sclf, 
                    param_grid=params, 
                    cv=5,
                    refit=True)

Changing

    def get_params(self, deep=True):
        """Return estimator parameter names for GridSearch support."""
        
        if not deep:
            return super(StackingCVClassifier, self).get_params(deep=False)
        else:
            out = self.named_classifiers.copy()
            for name, step in six.iteritems(self.named_classifiers):
                for key, value in six.iteritems(step.get_params(deep=True)):
                    out['%s__%s' % (name, key)] = value

            out.update(self.named_meta_classifier.copy())
            for name, step in six.iteritems(self.named_meta_classifier):
                for key, value in six.iteritems(step.get_params(deep=True)):
                    out['%s__%s' % (name, key)] = value
            return out

to

    def get_params(self, deep=True):
        """Return estimator parameter names for GridSearch support."""
        
        
        if not deep:
            return super(StackingCVClassifier, self).get_params(deep=False)
        else:
            out = self.named_classifiers.copy()
            for name, step in six.iteritems(self.named_classifiers):
                for key, value in six.iteritems(step.get_params(deep=True)):
                    out['%s__%s' % (name, key)] = value

            out.update(self.named_meta_classifier.copy())
            for name, step in six.iteritems(self.named_meta_classifier):
                for key, value in six.iteritems(step.get_params(deep=True)):
                    out['%s__%s' % (name, key)] = value
                    
            for key, value in six.iteritems(super(StackingCVClassifier, self).get_params(deep=False)):
                if key in ('classifiers', 'meta-classifier'):
                    continue
                else:
                    out['%s' % key] = value
                
            return out

in the StackingCVClassifier would solve the issue though. Similar modification should be made to the other stacking classifier, the stacking regressors, and the EnsembleVoteClassifier.

[jrbourbeau]

Hey @rasbt, I can work on this issue

[rasbt]

Oh yeah, thanks, that would be nice. I think it's not much, just the replacement like I outlined above (plus an entry in the changelog and inclusion in the unit tests :))