Class RFECV needs hard-coded n_features_to_select as a input parameter.

[KeithBrodie]

In scikit-learn 17.1 class RFECV uses a hard-coded 1 for n_features_to_select. For some classifiers with some parameter sets, the classifier will fail with only one feature. In addition, since RFECV takes some time, it would be helpful to allow the class to be initialized with a min_features_to_select with a value other than one that the user selects based on domain knowledge. One may not be interested in any RFECV solutions with less than, say five features. In this case the final four iterations of the recursive fit are wasted cycles.

The change is backward-compatible, anyone using RFECV without specifying min_features_to_select get's the argument default of one, the same behavior that's hard-coded today.

It's a trivial change to add it to the set of initialization parameters for the class. I use a customized version of sklearn 17.1 with this change. If I wasn't such a github noob I'd submit a pull or push or whatever it is.

Here's my code - modified RFECV init:

    def __init__(self, estimator, step=1, cv=None, scoring=None,
                 estimator_params=None, verbose=0, min_features_to_select=1):
        self.estimator = estimator
        self.step = step
        self.cv = cv
        self.scoring = scoring
        self.estimator_params = estimator_params
        self.min_features_to_select = min_features_to_select
        self.verbose = verbose

and in the fit() method:

        # Initialization
        cv = check_cv(self.cv, X, y, is_classifier(self.estimator))
        scorer = check_scoring(self.estimator, scoring=self.scoring)
        n_features = X.shape[1]
        n_features_to_select = self.min_features_to_select

[MechCoder]

Sounds like a good idea. Would really appreciate a Pull Request!

[MechCoder]

Btw, for what estimators does RFECV fail according to your comment?

[KeithBrodie]

My comment is too broad - the only one I know it fails for is XGBoost with the colsample_bytree parameter set non-zero. One could say that's as much a problem with XGBoost as it is with RFECV. My other point remains though, as it stands the function requires recursive elimination down to one feature in the sweep even if that's irrelevant to the solution of interest.

I'll try my hand at the pull request. This will be my first so if I mess up the works...

[MechCoder]

You can have a look at http://scikit-learn.org/stable/developers/contributing.html#contributing-code just in case you want to mess up a little less :-)

[grfiv]

I submitted pull request #8812 before I knew of this thread; it is pending my corrections to Travis CI's PEP 8 strictures. I was was motivated by the same XGB issue and I agree it's XGB's problem but since it is also my problem and also this feature seems useful more broadly so I suggested a change to RFE/RFECV.

The code snippet you include makes the change to n_features_to_select which I don't think really addresses the problem because I think most people would be happy with any number of features so long as they are the best.

I modified RFE along these lines (my new feature is currently called stop_at, although a recommendation has been made to rename it min_features):

# Elimination
while np.sum(support_) > n_features_to_select:
    # Remaining features
    if len(np.arange(n_features)[support_]) < self.stop_at:
        break

In addition, RFECV's calls to RFE must be modified