-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
by_2sd fails with model including interactions and non-standard contrasts #82
Comments
*WARNING: brain dump. I think this may be a much more general problem whenever there are complex/generated columns in the model (e.g. Suppose we have predictor variables (by which I mean variables that are present in the original data set) To be concrete, if we have
|
Thanks for this, Ben. I've been vaguely thinking of this problem (without really devoting any time to it) since this tweet by @leeper last month: https://twitter.com/thosjleeper/status/997127610072100864 Thanks, too and especially, for actually doing something about it. I'll look over your PR tomorrow. |
Thanks for merging the PR. Based on more conversation with my colleague JD, I think standardizing column-by-column may be fine (at least defensible and possibly even preferable to input-variable standardization). However, I think this warrants a bit more explication somewhere in the docs before closing this issue ... |
Two misc thoughts:
library(splines)
library(purrr)
fit <- lm(mpg ~ ns(hp, 4), mtcars)
mf <- model.frame(fit)
pred_classes <- flatten_chr(map(mf, class))
pred_classes
#> [1] "numeric" "ns" "basis" "matrix"
"matrix" %in% pred_classes
#> [1] TRUE Created on 2018-07-31 by the reprex |
For what it's worth, Schielzeth 2010 ("Simple means to improve the interpretability ..." Methods in Ecology and Evolution says
I haven't followed this up by looking at Gelman's paper (Statistics in Medicine, "Scaling regression inputs") yet. |
I am working with the development version of set.seed(123)
# models
mod1 <- stats::lm(data = iris, formula = Sepal.Length ~ Sepal.Width)
mod2 <- stats::lm(data = iris, formula = Sepal.Length ~ Species)
mod3 <- stats::lm(data = iris, formula = Sepal.Length ~ Sepal.Width * Species)
# broom outputs
df1 <- broom::tidy(mod1)
df2 <- broom::tidy(mod2)
df3 <- broom::tidy(mod3)
# trying to get the standardized estimates
dotwhisker::by_2sd(df1, mod1$model)
#> # A tibble: 2 x 6
#> term estimate std.error statistic p.value by_2sd
#> <chr> <dbl> <dbl> <dbl> <dbl> <lgl>
#> 1 (Intercept) 6.53 0.479 13.6 6.47e-28 TRUE
#> 2 Sepal.Width -0.195 0.135 -1.44 1.52e- 1 TRUE
dotwhisker::by_2sd(df2, mod2$model)
#> # A tibble: 3 x 6
#> term estimate std.error statistic p.value by_2sd
#> <chr> <dbl> <dbl> <dbl> <dbl> <lgl>
#> 1 (Intercept) 5.01 0.0728 68.8 1.13e-113 TRUE
#> 2 Speciesversicolor 0.93 0.103 9.03 8.77e- 16 TRUE
#> 3 Speciesvirginica 1.58 0.103 15.4 2.21e- 32 TRUE
dotwhisker::by_2sd(df3, mod3$model)
#> Error in `[[<-.data.frame`(`*tmp*`, paste0(first, ":", second), value = numeric(0)): replacement has 0 rows, data has 150 Created on 2018-12-09 by the reprex package (v0.2.1) |
tl;dr I agree that this is unclear, but I think it's essentially a documentation issue (requiring a documentation patch). At present,
i.e., it tries to get the model matrix and falls back to the model frame if it's unavailable. (I don't know whether I argued it in the thread above, but it's probably a good idea to return an informative error message that detects when columns are missing from |
PS re @fsolt's 31 July comment; matrix-valued input "variables" are one issue, but this problem is probably more general ... |
when there are interactions in the model,
by_2sd
tries to find the relevant columns of the model frame by parsing the names of the components in the interaction. This fails when contrasts have been set and the names are no longer as naively expected ...The example uses data from the
glmmTMB
package because it was what I was working with, but any model with an interaction (of two factors) should cause the same problem.I may work on the fix, but in the meantime I wanted to document this.
The text was updated successfully, but these errors were encountered: