Skip to content

performance 0.19.0

Latest

Choose a tag to compare

@strengejacke strengejacke released this 18 Oct 11:15
· 19 commits to main since this release
afb23a9

Changes

  • check_model() gets a show_smooth argument. With show_smooth = FALSE,
    the Linearity and Homogeneity of Variance panels draw neither the LOESS
    smoother nor the cell means, and the Influential Observations panel draws no
    LOESS smoother. This needs see newer than 0.14.2. Older versions ignore the
    setting.

  • check_model() and check_predictions() now work for multivariate models
    from package brms. Use the response argument to name the response to
    check. Without response, both functions give an error that lists the names
    of the responses (#908).

  • check_model() now shows the plot of influential observations for models
    from package glmmTMB with a gaussian, poisson, binomial, Gamma, or nbinom2
    family, and without zero-inflation or a dispersion formula. The plot uses hat
    values and a one-step approximation of Cook's distance, so the model is not
    refit. For other glmmTMB models, the plot is skipped, and with
    verbose = TRUE a message names the reason. For the same models,
    plot(check_outliers()) shows this plot with the Cook's distances from the
    refits (#95).

  • check_outliers() now computes the default threshold for Cook's distance as
    qf(0.5, p, n - p), as the documentation states. Here p is the number of
    fixed effects of the conditional model, and n is the number of
    observations in the fit. Before, p was the number of numeric variables in
    the data. If n - p is less than 1, method = "cook" gives an error.

  • check_outliers() gets a group argument for models from packages
    glmmTMB and lme4. It gives Cook's distance for each level of a grouping
    variable. The model is refit without each level in turn. With G the
    number of levels, levels above 4 / G are flagged (#95).

  • check_outliers() now supports models from package glmmTMB. Cook's
    distance is computed by refitting the model without each observation, and
    all data-based methods (like "zscore_robust" or "mahalanobis") work as
    well (#95).

  • model_performance() now supports models fitted to multiple imputed data
    sets with package lavaan.mi. The fit indices are pooled across imputations
    by lavaan::fitMeasures(), and pooling arguments such as pool.method can
    be passed through ... (#888).

  • check_overdispersion() now warns for models with a quasipoisson or
    quasibinomial family. These models already estimate a dispersion
    parameter, so their standard errors account for overdispersion. When over- or
    underdispersion is detected, the printed conclusion also says so (#734).

Bug fixes

  • check_collinearity() now gives the correct adjusted VIF (SE_factor) for
    factor terms in models with a factor response, such as logistic and ordinal
    models, and in mixed models with grouping factors. Before, these models got
    the square root of the VIF for every term, also for factors with more than
    two levels.

  • check_collinearity() now gives the correct VIFs for ordinal models from
    MASS::polr(), survey::svyolr(), ordinal::clm2(), ordinal::clmm2(), and
    rstanarm::stan_polr(). These models have one threshold for each outcome
    level except one, and no single intercept. Before, check_collinearity()
    dropped only the first column of the variance-covariance matrix as the
    intercept. The VIFs then included the thresholds. For MASS::polr(),
    survey::svyolr(), and rstanarm::stan_polr(), they also left out the first
    slope. Thus the VIFs were wrong.

  • Some models drop an aliased predictor, which is a linear combination of other
    predictors. For such ordinal::clm() and MASS::polr() models,
    check_collinearity() now warns that the model matrix is rank deficient. It
    also labels each VIF with its own term.

  • check_collinearity() uses a new method for the confidence intervals of
    VIF and tolerance. The method inverts the noncentral F distribution of the
    auxiliary regression of each term, as MBESS::ci.R2() does for fixed
    predictors (Kelley 2007). The old interval had an infinite upper limit for
    VIFs close to 1, and no upper limit for a VIF of exactly 1 (#597).

  • check_overdispersion() now uses the residual degrees of freedom of the
    model (df.residual()) for glm and fixest models, but not for svyglm()
    models (package survey). Before, it subtracted the number of coefficients
    from the number of observations. That was wrong for models with aliased
    coefficients and for fixest models with fixed effects. Now, the dispersion
    ratio of quasipoisson and quasibinomial glm models with aliased
    coefficients matches summary(model)$dispersion up to the fitting tolerance.
    It already did so for other stats::glm() models. Other classes that inherit
    from glm now also use their own df.residual(). For example, gam()
    models (package mgcv) use the effective degrees of freedom.

  • check_overdispersion() now gives correct results for models from package
    glmx with a negative binomial family, or with a binomial family and more
    than one trial. glmx has no method for Pearson residuals, so its stored
    deviance contributions were used before. Bernoulli models and other glmx
    families give an error.

  • If a model has no Pearson residuals, check_overdispersion() now gives an
    error. Before, it used the formula for Poisson models, which is wrong for
    other families.

  • The overdispersion plot of check_model() now uses the estimated dispersion
    for quasipoisson glm models. Before, it showed the variance of a Poisson
    model.

  • Fixed issue in check_predictions() for brms models with dec() response
    (Wiener and RT-choice families); the RT part of the model is now checked.

  • test_performance() now works for models from package lavaan. Before, it
    failed with an infinite recursion. It returns the Bayes factor and the
    likelihood-ratio test from lavaan::lavTestLRT(), and compares the models
    in order of their degrees of freedom. test_vuong() and test_wald() now give
    a clear error for lavaan models instead of the same recursion (#427).

  • check_collinearity() uses a new method for the confidence intervals of
    VIF and tolerance. The method inverts the noncentral F distribution of the
    auxiliary regression of each term, as MBESS::ci.R2() does for fixed
    predictors (Kelley 2007). The old interval had an infinite upper limit for
    VIFs close to 1, and no upper limit for a VIF of exactly 1 (#597).

  • check_overdispersion() now uses the residual degrees of freedom of the
    model (df.residual()) for glm and fixest models, but not for svyglm()
    models (package survey). Before, it subtracted the number of coefficients
    from the number of observations. That was wrong for models with aliased
    coefficients and for fixest models with fixed effects. Now, the dispersion
    ratio of quasipoisson and quasibinomial glm models with aliased
    coefficients matches summary(model)$dispersion up to the fitting tolerance.
    It already did so for other stats::glm() models. Other classes that inherit
    from glm now also use their own df.residual(). For example, gam()
    models (package mgcv) use the effective degrees of freedom.

  • check_overdispersion() now gives correct results for models from package
    glmx with a negative binomial family, or with a binomial family and more
    than one trial. glmx has no method for Pearson residuals, so its stored
    deviance contributions were used before. Bernoulli models and other glmx
    families give an error.

  • If a model has no Pearson residuals, check_overdispersion() now gives an
    error. Before, it used the formula for Poisson models, which is wrong for
    other families.

  • The overdispersion plot of check_model() now uses the estimated dispersion
    for quasipoisson glm models. Before, it showed the variance of a Poisson
    model.

  • Fixed issue in check_predictions() for brms models with dec() response
    (Wiener and RT-choice families); the RT part of the model is now checked.

  • test_performance() now works for models from package lavaan. Before, it
    failed with an infinite recursion. It returns the Bayes factor and the
    likelihood-ratio test from lavaan::lavTestLRT(), and compares the models
    in order of their degrees of freedom. test_vuong() and test_wald() now give
    a clear error for lavaan models instead of the same recursion (#427).

  • check_collinearity() now gives correct VIFs for multinomial models from
    package brglm2 (brmultinom(), and bracl() for adjacent-category
    models). It also gives correct VIFs for VGAM::vglm() models with the
    multinomial or cumulative family. Before, brmultinom() models and parallel
    bracl() models got VIFs for the wrong coefficients. Other bracl() models
    got no result, and vglm() models gave an error.
    Some models keep all coefficients for exactly collinear predictors, for
    example nnet::multinom(). For a singular correlation matrix of the
    coefficients, check_collinearity() now gives a warning and returns NULL.
    Before, it returned meaningless VIFs.

  • If you combine "cook" with a method that flags no observation, such as
    "zscore_robust", check_outliers() no longer fails. Cook's distance now
    also works for binomial models with only factor predictors. For glmmTMB
    models, the refits for Cook's distance now use the data given to the model.
    Thus, models with missing values, a subset, or weights and offsets outside
    the data get correct values. If a refit fails with the full model's
    estimates as start values, check_outliers() tries it again without them.

  • check_collinearity() now gives correct VIFs for multinomial models from
    packages nnet (multinom()) and mclogit (mblogit()). Before, only the
    first intercept was removed and coefficients were matched to terms by
    position, so the VIFs belonged to the wrong coefficients.
    Now each term gets one VIF, computed from its slopes across all outcome
    levels, so it does not depend on the reference level (#907).

  • check_model() now uses the value of show_ci that the user sets. Before,
    it ignored that value. It always hid the confidence bands for models with
    only categorical predictors and always showed them for other models
    (#642, #490).

  • check_collinearity() now returns interaction terms for brms models.
    Before, a model such as y ~ a * b gave no a:b row, and the VIFs of the
    other terms belonged to the wrong coefficients. Models with more than one
    intercept, such as ordinal models, also gave VIFs for the wrong
    coefficients.

  • compare_performance() no longer fails when only some of the models have
    LOOIC, for example a brms model and a glm (#738). It now computes
    stacking weights for the models that have LOOIC and gives NA for the
    others. A missing weight now prints as an empty cell, not as (>.999).

  • check_model() no longer fails for brms models that cannot be converted
    to a frequentist model. It now returns only the posterior predictive check
    and multicollinearity panels, and a message lists the panels that were
    skipped (#461).

  • check_collinearity() now labels terms correctly for brms models that use
    the 0 + Intercept formula syntax. Before, each VIF was shown under the name
    of the previous term and the last term was missing (#417).

  • r2_nakagawa() and icc() now include the random-slope variance for lme4
    models with uncorrelated random slopes, such as (x || group). The fix is in
    insight::get_variance(). For these models, a shift of a predictor changes
    the values, and the documentation of both functions now says so (#428).

  • model_performance() for lavaan models now passes further arguments in
    ... to lavaan::fitMeasures() by name. Before, any extra argument other
    than estimator caused an error, even an argument that fitMeasures()
    accepts, such as fm_args (fm.args in lavaan before 0.7).

  • model_performance() for lavaan models no longer errors when
    lavaan::fitMeasures() does not return an index, for example the
    loglikelihood, AIC, and BIC of models with ordinal indicators. These
    indices are now NA.

  • check_overdispersion() now computes the Pearson chi-squared statistic from
    the model's own Pearson residuals whenever it does not use simulated
    residuals. Results change, and are now correct, for binomial models with
    more than one trial per observation (a cbind() response, or proportions
    with weights) and for models with prior weights. Before, the statistic
    used unweighted Poisson-style residuals and could be far off.

  • check_model() now correctly returns the random-effects normality panel for
    mixed models that specify random slopes using split terms such as
    (1 | group) + (0 + slope | group) (including || expansions, #845).

  • check_singularity() for models from package nlme no longer errors for
    nested random effects with more than one grouping level and now applies the
    tolerance check to the absolute values of the variance-covariance diagonal.

  • check_collinearity() no longer drops interaction terms when fallback term
    assignment is used (for example, in some brmsfit models, #936).

  • Preserve method–threshold alignment in print() method for check_outliers()
    for multi-method input.

  • r2_tjur(), r2_somers(), r2_efron() and r2_mckelvey() now warn and
    return NULL for binomial models that are not Bernoulli models, like r2().
    Examples are a proportion response with weights and a cbind() response.
    Before, these functions returned inaccurate values or failed with an unclear
    error (#165).

  • The influential observations plot of check_model() and check_outliers()
    for linear mixed models from package lme4 now shows standardized residuals
    instead of raw residuals. Before this fix, points fell outside the Cook's
    distance lines but were not flagged as outliers (#857).

  • The Cook's distance lines in the influential observations plot of
    check_model() and check_outliers() now use the threshold of
    check_outliers(), including a user-supplied threshold. Before this fix,
    linear models always used a threshold from the model's F statistic, so the
    lines did not match the flagged outliers (#916).