R caret: rfe nnet “undefined columns selected”

[david-machinelearning]

As reported on SO (http://stackoverflow.com/questions/39418075/r-caret-rfe-nnet-undefined-columns-selected):

I am running into problems when applying recursive feature selection to nnet models with caret::rfe; I get the following error message:

Error in { : task 1 failed - "undefined columns selected"

The actual task is more complex than the following example, but I am confident that this is a similar problem:

library(caret)

rfe(x = iris[,1:3],
         y = iris[,4]/max(iris[,4]),
         sizes = c(2), 
         method="nnet",
         rfeControl = rfeControl(functions = caretFuncs) 
         )

I know this error can occur when trying to select more features than there are available in x (e.g. see http://stats.stackexchange.com/questions/18362/odd-error-with-caret-function-rfe), but this does not seem to be the problem here. I also ran very similar calls in earlier versions of caret, without this problem occurring.

I use R 3.3.1 and caret 6.0.71.

Thank you very much for your help.

EDIT: I went through the archived versions of caret and found that the example code is working in caret versions <= 6.0.62.

[topepo]

It looks like a bug. That code hasn't been touched in a while so I'm not sure why it would show up as an error now unless the underlying package changed. Either way, I'll make sure that it gets fixed.

Thanks,

Max

[topepo]

Please take a look at those changes. The test case that you gave me works but it would be good to test it in the context of what you were working on (assuming it wasn't the irises)

[david-machinelearning]

Dear Max,
I just used the development version of caret for my initial problem; the issue does not occur any more.
Thanks for looking into this!
David

[topepo]

No problem. Thanks!

[eliasmccosta]

Dear Max,
I had the same problem using the function (functions = lmFuncs), but it worked fine when I used the functions gamFuncs and rfFuncs. I believe it's a bug in the lmFuncs function. My version of caret package is the last one 6.0-77.

[cdeterman]

Same issue for me with lmFuncs in caret_6.0-77. Also just tested with caret_6.0-78.

[vors]

Same for me

[sfeuerriegel]

Here as well

[ghost]

Similar issue

[seymakalay]

hi everyone,
I am running MARS recursive model for 3 different outcomes:2 of them work as the same way below: but the last one gives me error "undefined columns selected" yet the settings for all the 3 different outcomes are the same for MARS,

Can anyone help??
many thanks in advance,,

inTrainingSet_aml <- createDataPartition(combined_AML$index.oninvestment ,p=.80,list=0)
combined_AML_Train <- combined_AML[inTrainingSet_aml,]
combined_AML_Test <- combined_AML[-inTrainingSet_aml,]
##############
#set xx and yy
##############
xxx_I1 <- combined_AML_Train [,c(7:17,21:42,44)]
yyy_I1 <- combined_AML_Train$index.oninvestment

predVars.I1 <- names(combined_AML)[!(names(combined_AML) %in% c( "index.oninvestment",
"some more class columns which I dont wanna consider" ))]
varSeq <- seq(1,length(predVars.I1)-1,by=2)
indexx <- createMultiFolds(index.oninvestment,times = 5)

marsFuncs <- caretFuncs
marsFuncs$summary <- defaultSummary

ctrl.rcrsive.I1.mars <- rfeControl(method = "repeatedCV",
repeats = 5, verbose = T,
functions = marsFuncs,
index = indexx )
set.seed(87)
marsRFE.I1 <- rfe(xxx_I1,yyy_I1,
sizes = varSeq,
rfeControl = ctrl.rcrsive.I1.mars,
method="earth", tuneLength=10,
preProcess = c("knnImpute","center","scale"))
marsRFE.I1

[topepo]

@seymakalay

Adding to a closed issue is not likely to get a response (especially since this seems unrelated to the issue that was discussed here). Most developers triage open issues and these comments fall between the cracks since nobody is looking at them.

If you question is about having multivariate outcomes, caret doesn't support that.

If not, please start a new issue and follow the issue template to help get this answered. Referencing a data set that we don't have access to doesn't help.