Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CRAN found a revdep problem with the insight package (details below) #679

Closed
strengejacke opened this issue Nov 9, 2022 · 3 comments · Fixed by #680
Closed

CRAN found a revdep problem with the insight package (details below) #679

strengejacke opened this issue Nov 9, 2022 · 3 comments · Fixed by #680
Assignees
@strengejacke
Labels
get_predicted Function specific issues High priority 🏃 This issue should be addressed soon

Comments

@strengejacke
Copy link
Member

strengejacke commented Nov 9, 2022 •
edited
Loading

CRAN found a revdep problem with the insight package (details below). @strengejacke, I think you are the maintainer. I need to be in an online meeting for most of today, but will try get to this in the evening if someone else hasn't already.

Package: insight
Check: tests
New result: ERROR
Running ‘testthat.R’ [82s/82s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
if (require("testthat")) {
   +   library(insight)
   + 
   +   is_dev_version <- length(strsplit(packageDescription("insight")$Version, "\\.")[[1]]) > 3
   + 
   +   if (is_dev_version) {
   +     Sys.setenv("RunAllinsightTests" = "yes")
   +   } else {
   +     Sys.setenv("RunAllinsightTests" = "no")
   +   }
   +   si <- [Sys.info](http://sys.info/)()
   + 
   +   osx <- tryCatch(
   +     {
   +       if (!is.null(si["sysname"])) {
   +         si["sysname"] == "Darwin" || grepl("^darwin", R.version$os)
   +       } else {
   +         FALSE
   +       }
   +     },
   +     error = function(e) {
   +       FALSE
   +     }
   +   )
   + 
   +   solaris <- tryCatch(
   +     {
   +       if (!is.null(si["sysname"])) {
   +         grepl("SunOS", si["sysname"], ignore.case = TRUE)
   +       } else {
   +         FALSE
   +       }
   +     },
   +     error = function(e) {
   +       FALSE
   +     }
   +   )
   + 
   +   # disable / enable if needed
   +   if (.Platform$OS.type == "unix" && is_dev_version) {
   +     Sys.setenv("RunAllinsightStanTests" = "yes")
   +   } else {
   +     Sys.setenv("RunAllinsightStanTests" = "no")
   +   }
   + 
   +   # if (!osx && !solaris) {
   +   #   test_check("insight")
   +   # }
   + 
   +   test_check("insight")
   + }
   Loading required package: testthat
   boundary (singular) fit: see help('isSingular')
   Some of the variables were in matrix-format - probably you used
     `scale()` on your data?
     If so, and you get an error, please try `datawizard::standardize()` to
     standardize your data.
   Confidence intervals are not yet supported for models of class `mlm`.

   Iteration 1 - deviance = 39.74973 - criterion = 0.8590917
   Iteration 2 - deviance = 10.50328 - criterion = 2.758244
   Iteration 3 - deviance = 9.231325 - criterion = 0.1363107
   Iteration 4 - deviance = 9.227742 - criterion = 0.0003840654
   Iteration 5 - deviance = 9.227742 - criterion = 3.446463e-09
   converged
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.

   Re-fitting to get Hessian


   Re-fitting to get Hessian

   boundary (singular) fit: see help('isSingular')
   boundary (singular) fit: see help('isSingular')
   [ FAIL 4 | WARN 51 | SKIP 16 | PASS 2516 ]

   ══ Skipped tests ═══════════════════════════════════════════════════════════════
   • On CRAN (6)
   • isFALSE(run_stan) is TRUE (5)
   • works interactively (5)

   ══ Failed tests ════════════════════════════════════════════════════════════════
   ── Failure ('test-glmmTMB.R:927'): get_predicted ───────────────────────────────
   `x` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...
   ── Failure ('test-glmmTMB.R:928'): get_predicted ───────────────────────────────
   `y` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...
   ── Failure ('test-glmmTMB.R:936'): get_predicted ───────────────────────────────
   `x` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...
   ── Failure ('test-glmmTMB.R:937'): get_predicted ───────────────────────────────
   `y` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...

   [ FAIL 4 | WARN 51 | SKIP 16 | PASS 2516 ]
   Error: Test failures
   Execution haltedPackage: insight
Check: tests
New result: ERROR
Running ‘testthat.R’ [82s/82s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
if (require("testthat")) {
   +   library(insight)
   + 
   +   is_dev_version <- length(strsplit(packageDescription("insight")$Version, "\\.")[[1]]) > 3
   + 
   +   if (is_dev_version) {
   +     Sys.setenv("RunAllinsightTests" = "yes")
   +   } else {
   +     Sys.setenv("RunAllinsightTests" = "no")
   +   }
   +   si <- [Sys.info](http://sys.info/)()
   + 
   +   osx <- tryCatch(
   +     {
   +       if (!is.null(si["sysname"])) {
   +         si["sysname"] == "Darwin" || grepl("^darwin", R.version$os)
   +       } else {
   +         FALSE
   +       }
   +     },
   +     error = function(e) {
   +       FALSE
   +     }
   +   )
   + 
   +   solaris <- tryCatch(
   +     {
   +       if (!is.null(si["sysname"])) {
   +         grepl("SunOS", si["sysname"], ignore.case = TRUE)
   +       } else {
   +         FALSE
   +       }
   +     },
   +     error = function(e) {
   +       FALSE
   +     }
   +   )
   + 
   +   # disable / enable if needed
   +   if (.Platform$OS.type == "unix" && is_dev_version) {
   +     Sys.setenv("RunAllinsightStanTests" = "yes")
   +   } else {
   +     Sys.setenv("RunAllinsightStanTests" = "no")
   +   }
   + 
   +   # if (!osx && !solaris) {
   +   #   test_check("insight")
   +   # }
   + 
   +   test_check("insight")
   + }
   Loading required package: testthat
   boundary (singular) fit: see help('isSingular')
   Some of the variables were in matrix-format - probably you used
     `scale()` on your data?
     If so, and you get an error, please try `datawizard::standardize()` to
     standardize your data.
   Confidence intervals are not yet supported for models of class `mlm`.

   Iteration 1 - deviance = 39.74973 - criterion = 0.8590917
   Iteration 2 - deviance = 10.50328 - criterion = 2.758244
   Iteration 3 - deviance = 9.231325 - criterion = 0.1363107
   Iteration 4 - deviance = 9.227742 - criterion = 0.0003840654
   Iteration 5 - deviance = 9.227742 - criterion = 3.446463e-09
   converged
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.
   Confidence intervals are not yet supported for models of class
     `multinom`.

   Re-fitting to get Hessian


   Re-fitting to get Hessian

   boundary (singular) fit: see help('isSingular')
   boundary (singular) fit: see help('isSingular')
   [ FAIL 4 | WARN 51 | SKIP 16 | PASS 2516 ]

   ══ Skipped tests ═══════════════════════════════════════════════════════════════
   • On CRAN (6)
   • isFALSE(run_stan) is TRUE (5)
   • works interactively (5)

   ══ Failed tests ════════════════════════════════════════════════════════════════
   ── Failure ('test-glmmTMB.R:927'): get_predicted ───────────────────────────────
   `x` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...
   ── Failure ('test-glmmTMB.R:928'): get_predicted ───────────────────────────────
   `y` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...
   ── Failure ('test-glmmTMB.R:936'): get_predicted ───────────────────────────────
   `x` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...
   ── Failure ('test-glmmTMB.R:937'): get_predicted ───────────────────────────────
   `y` (`actual`) not equal to `z` (`expected`).

        actual       | expected                     
    [1] 0.2400130397 - 0.3747267975 [1]             
    [2] 0.9726394192 - 1.1037499976 [2]             
    [3] 0.2400130397 - 0.3747267975 [3]             
    [4] 0.4613929713 - 0.5692223933 [4]             
    [5] 0.2400130397 - 0.3747267975 [5]             
    [6] 0.6362097213 - 0.6854377778 [6]             
    [7] 0.3344547486 - 0.4168940552 [7]             
    [8] 0.0059284577 - 0.0178449327 [8]             
    [9] 0.0902188470 - 0.1367765053 [9]             
   [10] 0.9726394192 - 1.1037499976 [10]            
    ... ...            ...          and 240 more ...

   [ FAIL 4 | WARN 51 | SKIP 16 | PASS 2516 ]
   Error: Test failures
   Execution halted

Originally posted by @mebrooks in glmmTMB/glmmTMB#873 (comment)
@strengejacke strengejacke added High priority 🏃 This issue should be addressed soon get_predicted Function specific issues labels Nov 9, 2022
@strengejacke strengejacke changed the title CRAN found a revdep problem with the insight package (details below). @strengejacke, I think you are the maintainer. I need to be in an online meeting for most of today, but will try get to this in the evening if someone else hasn't already. CRAN found a revdep problem with the insight package (details below) Nov 9, 2022
@DominiqueMakowski
Copy link
Member

Seems related to easystats/modelbased#217

@strengejacke
Copy link
Member Author

Yes, they changed something in the truncated_poisson family. I found out following (copying my response from the related glmmTMB issue):

I think the problem is that for zero-inflation models, we assume that the predicted response is mu * (1 - zi_prob). We also compute predictions on the link-scale, then backtransform. However, this behaviour seems to have changed:

library(glmmTMB)
library(insight)
data("fish")
  m1 <- glmmTMB(
  count ~ child + camper + (1 | persons),
  ziformula = ~ child + camper + (1 | persons),
  data = fish,
  family = truncated_poisson()
)

predict(m1, type = "response") |> head()
#> [1] 0.3747268 1.1037500 0.3747268 0.5692224 0.3747268 0.6854378

(link_inverse(m1)(predict(m1, type = "link")) * (1 - as.vector(predict(m1, type = "zprob")))) |> head()
#> [1] 0.2400130 0.9726394 0.2400130 0.4613930 0.2400130 0.6362097

linv <- family(m1)$linkinv
(linv(predict(m1, type = "link")) * (1 - as.vector(predict(m1, type = "zprob")))) |> head()
#> [1] 0.2400130 0.9726394 0.2400130 0.4613930 0.2400130 0.6362097

My question is, why is predicting the response no longer equal to predicting the count model on the link-scale, back-transform via link-inverse and multiply with the predicted zero-inflation-probability?

@strengejacke strengejacke self-assigned this Nov 9, 2022
@strengejacke strengejacke mentioned this issue Nov 9, 2022
Merged
@strengejacke
Copy link
Member Author

This issue also might affect the way we calculate CIs for truncated families (via the simulations)

@strengejacke strengejacke mentioned this issue Nov 9, 2022
Open
@strengejacke strengejacke mentioned this issue Oct 6, 2023
Open
@strengejacke strengejacke mentioned this issue May 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@strengejacke strengejacke

Labels
get_predicted Function specific issues High priority 🏃 This issue should be addressed soon
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Revdep issues with glmmTMB easystats/insight
2 participants
@DominiqueMakowski @strengejacke