Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test frame as tibble - wrong calculations for variable importance and confusing warning messages #15

Closed
kpmakowski opened this issue Apr 24, 2018 · 2 comments
Labels
bug 💣 Bug to fix

Comments

@kpmakowski
Copy link

If you do have test frame as tibble (easy to get when using tidyverse)

For calculation of variable importance you get the same values for full_model and all variables except baseline, which is obviously wrong.
variable dropout_loss label
1 full_model 284.9159 lm
2 construction.year 284.9159 lm
3 surface 284.9159 lm
4 floor 284.9159 lm
5 no.rooms 284.9159 lm
6 district 284.9159 lm
7 baseline 1261.6643 lm

For single_variable calculations you get following warning (only), however output is of limited value.
Warning message:
In if (class(explainer$data[, variable]) == "factor" & type != "factor") { :
the condition has length > 1 and only the first element will be used

Casting tibble to regular data.frame solves the issue. Having training data as tibble seems not to have an impact on calculations at all.

`apartmentsTest_tibble <- apartmentsTest %>% as_tibble()

model_liniowy <- lm(m2.price ~ construction.year + surface + floor + no.rooms + district, data = apartments)

explainer_lm <- explain(model_liniowy, data = apartmentsTest_tibble[,2:6], y = apartmentsTest_tibble$m2.price)

vi_lm <- variable_importance(explainer_lm, loss_function = loss_root_mean_square)
vi_lm

sv_lm <- single_variable(explainer_lm, variable = "construction.year", type = "pdp")`

@pbiecek pbiecek added the bug 💣 Bug to fix label Apr 24, 2018
@pbiecek
Copy link
Member

pbiecek commented Apr 24, 2018

Thanks for pointing this out, will fix soon

pbiecek added a commit that referenced this issue Apr 24, 2018
pbiecek added a commit that referenced this issue Apr 24, 2018
@pbiecek
Copy link
Member

pbiecek commented Apr 28, 2018

fixed in 59e4295

@pbiecek pbiecek closed this as completed Apr 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 💣 Bug to fix
Projects
None yet
Development

No branches or pull requests

2 participants