ALE doesn't preserve factor level order in the output

[hbaniecki]

Consider the following code:

library("DALEX")
library("randomForest")

d2 <- titanic_imputed
levels(d2$gender) <- c("male" ,"female")
model_titanic_rf <- randomForest(survived ~ .,  data = d2)

explain_titanic_rf <- explain(model_titanic_rf,
                              data = d2[,-8],
                              y = d2[,8])

selected_passangers <- select_sample(d2, n = 100)
cp_rf <- ceteris_paribus(explain_titanic_rf, selected_passangers)

ale <- aggregate_profiles(cp_rf, variables = "gender", type = "accumulated", variable_type = "categorical")

ale$`_x_`
levels(d2$gender)
plot(ale)

I deliberately changed the factor levels to showcase the problem. _x_ column should probably be sorted on d2$gender levels (they matter in computation). The same may apply to all aggregate_profiles outputs.

[hbaniecki]

Ceteris Paribus does that:

library("DALEX")
# smaller data, quicker example
titanic_small <- select_sample(titanic_imputed, n = 500, seed = 1313)
levels(titanic_small$gender) <- c("male", "female")
# build a model
model_titanic_glm <- glm(survived ~ gender + age + fare,
                         data = titanic_small,
                         family = "binomial")

explain_titanic_glm <- explain(model_titanic_glm,
                               data = titanic_small[,-8],
                               y = titanic_small[,8],
                               verbose = FALSE)

cp_rf <- ceteris_paribus(explain_titanic_glm, titanic_small[1,])
levels(cp_rf$gender)

[pbiecek]

Not that easy, as _x_ has levels for all factor variables.
And these levels are stored as characters not as factors. Characters do not have order.

Maybe the easiest solution would to add some postprocessing that will convert characters into factors with an ordered derived from the input.

[hbaniecki]

Yes, this should be the best solution.

[pbiecek]

CP is working fine

> levels(cp_rf$gender)
[1] "male"   "female"

it's a problem with aggregate_profiles

[hbaniecki]

Sure, I meant that it is working fine. We can just sort _x_ column in corresponding groups, but have in mind that not all of the columns have to be factors (not factors can have unique(x) order, as it's probably used in computation). NVM. maybe they have to be factors.

library("DALEX")
library("randomForest")

d2 <- titanic_imputed
levels(d2$gender) <- c("male" ,"female")
model_titanic_rf <- randomForest(survived ~ .,  data = d2)

explain_titanic_rf <- explain(model_titanic_rf,
                              data = d2[,-8],
                              y = d2[,8])

selected_passangers <- select_sample(d2, n = 100)
cp_rf <- ceteris_paribus(explain_titanic_rf, selected_passangers)

ale <- aggregate_profiles(cp_rf, variables = c("gender","embarked"), type = "accumulated", variable_type = "categorical")

ale$`_x_`
c(levels(d2$gender),levels(d2$embarked))

levels(c("b", "a")) #NULL
unique(c("b", "a"))