Pre-processing change in caret 6.0-58 cause the Adaptive Resampling error

[elephann]

I noticed a new fix of pre-processing in the latest version caret 6.0-58, but this new fix can cause the Adaptive Resampling error.Here is the code:

set.seed(1)
class_tr <- twoClassSim(200)
class_te <- twoClassSim(200)

set.seed(1)
reg_tr <- SLC14_1(200)
reg_te <- SLC14_1(200)

classification

set.seed(2)
lda_mod <- train(Class ~ ., data = class_tr,

•                  method    = "lda",
  

•                  preProc   = c("center", "scale"),
  

•                  trControl = trainControl(method = "adaptive_cv", classProbs = TRUE),
  

•                  adaptive  = list(min         = 10,
  

•                                   alpha       = 0.05,
  

•                                   method      = "gls",
  

•                                   complete    = TRUE))
  

  Error in { : task 1 failed - "argument is not interpretable as logical"

set.seed(2)
rf_mod <- train(Class ~ ., data = class_tr,

•                  method    = "rf",
  

•                  preProc   = c("center", "scale"),
  

•                  trControl = trainControl(method = "adaptive_cv", classProbs = TRUE),
  

•                  adaptive  = list(min         = 10,
  

•                                   alpha       = 0.05,
  

•                                   method      = "BT",
  

•                                   complete    = TRUE))
  

  Error in { : task 1 failed - "argument is not interpretable as logical"

While the old version 6.0-57 won't throw out this error.

Besides, when using the Adaptive Resampling methods, some time the results shows: "the x parameter filtering failed. "Though with the error, it seems predict still work, what does that mean?

[topepo]

A couple of things:

• there are no tuning parameters for lda so this won't work (there are none to eliminate)
• for rf_mod, the adaptive argument should be part of trainControl. The code above is mangled but it looks like it is passed as an argument to train
• min = 10 cannot work when you are only doing 10-fold CV

The last item shows that you didn't test the exact code before submitting. Please do this with a reproducible example next time.

However, there is a bug and it doesn't have anything to do with preProcess but with the changes to savePredictions that produce the "argument is not interpretable as logical" note. I'll fix that.

Thanks,

Max

[topepo]

It looks like things are fixed now:

> library(caret)
> 
> set.seed(1)
> dat <- twoClassSim(100)
> 
> mod1 <- train(Class ~ ., data = dat, 
+               method = "rf",
+               preProc = c("center", "scale"),
+               trControl = trainControl(method = "adaptive_cv", 
+                                        classProbs = TRUE,
+                                        verboseIter = TRUE,
+                                        adaptive = list(min = 5,
+                                                        alpha = 0.05,
+                                                        method = "BT",
+                                                        complete = TRUE)))
+ Fold01.Rep1: mtry= 2 
- Fold01.Rep1: mtry= 2 
+ Fold01.Rep1: mtry= 8 
- Fold01.Rep1: mtry= 8 
+ Fold01.Rep1: mtry=15 
- Fold01.Rep1: mtry=15 
+ Fold02.Rep1: mtry= 2 
- Fold02.Rep1: mtry= 2 
+ Fold02.Rep1: mtry= 8 
- Fold02.Rep1: mtry= 8 
+ Fold02.Rep1: mtry=15 
- Fold02.Rep1: mtry=15 
+ Fold03.Rep1: mtry= 2 
- Fold03.Rep1: mtry= 2 
+ Fold03.Rep1: mtry= 8 
- Fold03.Rep1: mtry= 8 
+ Fold03.Rep1: mtry=15 
- Fold03.Rep1: mtry=15 
+ Fold04.Rep1: mtry= 2 
- Fold04.Rep1: mtry= 2 
+ Fold04.Rep1: mtry= 8 
- Fold04.Rep1: mtry= 8 
+ Fold04.Rep1: mtry=15 
- Fold04.Rep1: mtry=15 
+ Fold05.Rep1: mtry= 2 
- Fold05.Rep1: mtry= 2 
+ Fold05.Rep1: mtry= 8 
- Fold05.Rep1: mtry= 8 
+ Fold05.Rep1: mtry=15 
- Fold05.Rep1: mtry=15 
o no models eliminated; 3 remain
+ Fold06.Rep1: mtry= 2 
- Fold06.Rep1: mtry= 2 
+ Fold06.Rep1: mtry= 8 
- Fold06.Rep1: mtry= 8 
+ Fold06.Rep1: mtry=15 
- Fold06.Rep1: mtry=15 
o no models eliminated; 3 remain
+ Fold07.Rep1: mtry= 2 
- Fold07.Rep1: mtry= 2 
+ Fold07.Rep1: mtry= 8 
- Fold07.Rep1: mtry= 8 
+ Fold07.Rep1: mtry=15 
- Fold07.Rep1: mtry=15 
o no models eliminated; 3 remain
+ Fold08.Rep1: mtry= 2 
- Fold08.Rep1: mtry= 2 
+ Fold08.Rep1: mtry= 8 
- Fold08.Rep1: mtry= 8 
+ Fold08.Rep1: mtry=15 
- Fold08.Rep1: mtry=15 
o no models eliminated; 3 remain
+ Fold09.Rep1: mtry= 2 
- Fold09.Rep1: mtry= 2 
+ Fold09.Rep1: mtry= 8 
- Fold09.Rep1: mtry= 8 
+ Fold09.Rep1: mtry=15 
- Fold09.Rep1: mtry=15 
o no models eliminated; 3 remain
+ Fold10.Rep1: mtry= 2 
- Fold10.Rep1: mtry= 2 
+ Fold10.Rep1: mtry= 8 
- Fold10.Rep1: mtry= 8 
+ Fold10.Rep1: mtry=15 
- Fold10.Rep1: mtry=15 
o no models eliminated; 3 remain
Aggregating results
Selecting tuning parameters
Fitting mtry = 2 on full training set

At least I know that someone is using it =]

Thanks,

Max

[elephann]

Thank you so much! Max!
I am new user of R and learn it myself. Recently,I am trying to apply predicative methods in finance. Sorry for the mangled code! I do appreciate your help and your excellent work.
With my best regards!
Daniel