What does ranger do with new factor levels in prediction?

[mayer79]

Hello rangers

I was recently stumbling over the error message

Error in predict.ranger.forest(forest, data, predict.all, seed, num.threads, :
Missing values in data.

It was the "classic" problem of having a new fector level in a categorical predictor during prediction which seems to happen if respect.unordered.factors = ["order"/TRUE] only. For "partition" and "ignore"/FALSE, there is no such message.

I think the behaviour in the cases "order" and "ignore" (FALSE) is clear although the error message for "order" could be more specific like "new or unknown factor levels in regressor". But what does ranger do in the last case respect.unordered.factors = "partition" (no error)?

Below the small example for test:

# All possible two-partitions
fit <- ranger(Sepal.Width ~ Species, data = iris, write.forest = TRUE, respect.unordered.factors = "partition")
predict(fit, data.frame(Species = ""))$predictions

# Ordered by proportion of second class (respect.unordered.factors = TRUE)
fit <- ranger(Sepal.Width ~ Species, data = iris, write.forest = TRUE, respect.unordered.factors = "order")
predict(fit, data.frame(Species = ""))$predictions

# Factors are considered ordered (respect.unordered.factors = FALSE)
fit <- ranger(Sepal.Width ~ Species, data = iris, write.forest = TRUE, respect.unordered.factors = "ignore")
predict(fit, data.frame(Species = ""))$predictions

[mnwright]

Both things are on my TODO list for a while:

1. Fix error for respect.unordered.factors = "order"
2. How to handle new levels in general?

On 2.: In randomForest all new factor levels go to the left. Currently the same is done in ranger. However, we should change this. Any better ideas than just assign them randomly?

[mayer79]

Oh I see. So I guess the same strategy (always to the left) is also used when calculating the OOB predictions. Maybe you can leave it like that until we have strategies of handling missing values in predictors? Then you could start to offer an option like newFactorLevels = c("left", "right", "missing") or so.

[mnwright]

Error fixed in #120.

[ghost]

In randomForest all new factor levels go to the left.

i'm not sure about this
randomForest.predict returns error "new factor levels not present in the training data" when it is faced with unseen levels.

[mnwright]

Yes, you are right. I was checking for factor levels being present in the levels but not in the data. However, I've just checked that again and it was changed in a recent version of randomForest: They are assigned randomly since version 4.6-10.

[ghost]

I used this datasets https://cloud.mail.ru/public/5XwH/oR6FP8qwq and got error in randomForest 4.6.12 library(randomForest) randomForest 4.6-12 Type rfNews() to see new features/changes/bug fixes. data <- read.csv2("C:/Trees/Exp.csv") data$x1 <- as.factor(data$x1) data$x2 <- as.factor(data$x2) data$x3 <- as.factor(data$x3) data$x4 <- as.factor(data$x4) data$x6 <- as.factor(data$x6) data$y <- as.factor(data$y) model <- randomForest(y ~., data, seed=100) newdata <- read.csv2("C:/Trees/Exp_new.csv") newdata$x1 <- as.factor(newdata$x1) newdata$x2 <- as.factor(newdata$x2) newdata$x3 <- as.factor(newdata$x3) newdata$x4 <- as.factor(newdata$x4) newdata$x6 <- as.factor(newdata$x6) score <- predict(model, newdata, type="prob") Ошибка в predict.randomForest(model, newdata, type = "prob") : New factor levels not present in the training data