Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test frame as tibble - wrong calculations for variable importance and confusing warning messages #15

Closed
kpmakowski opened this issue Apr 24, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@kpmakowski
Copy link

commented Apr 24, 2018

If you do have test frame as tibble (easy to get when using tidyverse)

For calculation of variable importance you get the same values for full_model and all variables except baseline, which is obviously wrong.
variable dropout_loss label
1 full_model 284.9159 lm
2 construction.year 284.9159 lm
3 surface 284.9159 lm
4 floor 284.9159 lm
5 no.rooms 284.9159 lm
6 district 284.9159 lm
7 baseline 1261.6643 lm

For single_variable calculations you get following warning (only), however output is of limited value.
Warning message:
In if (class(explainer$data[, variable]) == "factor" & type != "factor") { :
the condition has length > 1 and only the first element will be used

Casting tibble to regular data.frame solves the issue. Having training data as tibble seems not to have an impact on calculations at all.

`apartmentsTest_tibble <- apartmentsTest %>% as_tibble()

model_liniowy <- lm(m2.price ~ construction.year + surface + floor + no.rooms + district, data = apartments)

explainer_lm <- explain(model_liniowy, data = apartmentsTest_tibble[,2:6], y = apartmentsTest_tibble$m2.price)

vi_lm <- variable_importance(explainer_lm, loss_function = loss_root_mean_square)
vi_lm

sv_lm <- single_variable(explainer_lm, variable = "construction.year", type = "pdp")`

@pbiecek pbiecek added the bug label Apr 24, 2018

@pbiecek

This comment has been minimized.

Copy link
Owner

commented Apr 24, 2018

Thanks for pointing this out, will fix soon

pbiecek added a commit that referenced this issue Apr 24, 2018

pbiecek added a commit that referenced this issue Apr 24, 2018

@pbiecek

This comment has been minimized.

Copy link
Owner

commented Apr 28, 2018

fixed in 59e4295

@pbiecek pbiecek closed this Apr 28, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.