possible test failure with development version of lme4

[bbolker]

Hi,

I'm in the process of sending a new version of lme4 to CRAN. It looks like this might break variancePartition tests, although I'm having a little bit of trouble digging in and finding out exactly what the issue is (I have a development version of R, which gives me some challenges in installing Bioconductor packages).

Possibly? because a previously OK fit becomes singular? Related to lme4/lme4#669 ?

update: something as simple as this breaks:

library(variancePartition)
data(varPartData)
form <- ~ Age + (1|Individual) + (1|Tissue) + (1|Batch)
varPart <- fitExtractVarPartModel( geneExpr, form, info )

Dividing work into 10 chunks...
boundary (singular) fit: see help('isSingular')
Error: BiocParallel errors
1 remote errors, element index: 1
0 unevaluated and other errors
first remote error:
Error in run_model_check_mixed(fit, showWarnings, dream, colinearityCutoff, : boundary (singular) fit: see help('isSingular')

[GabrielHoffman]

Thanks fo the notice.

1. What version of variancePartition?
2. Can you point me to the new lme4 version, so I can reproduce the error

Gabriel

[bbolker]

I actually think I've figured this out/am partway through fixing it. The problem is that lmer no longer skips the singular-fit check when calc.derivs is FALSE in lmerControl. See bbolker@500a400 (sorry about all the whitespace changes!!)

New version of lme4 is at https://github.com/lme4/lme4/ (remotes::install_github("lme4/lme4")

[bbolker]

Here's a patch (without the whitespace problems ...)

vp_patch.txt

[GabrielHoffman]

I confirmed that the patch fixes the error.

In testing out my new version, I came across this issue with lme4 1.1.29. I noticed it in varPartConfInf(), but it is an issue with lme4::bootMer(). It looks like in computing bootstraps, one of the models has a variance component of zero so it is singular. This is reported as warning when run, and then an error when the result is printed. This is confusing, since statistically this is common enough in bootstrapping.

In lme4 1.1.27.1, it is only a message, not an error and warning.

Here is a minimal reproducible example using must lme4:

library(variancePartition)
library(lme4)

data(varPartData)

y = c(-7.67,-19.113,-8.57,-9.186,-8.309,-13.097,-7.595,-18.354,-11.205,-12.517,-4.903,-8.199,-6.605,-11.924,-9.538,-6.165,-14.9,-8.926,-9.321,-13.713,-9.393,-12.575,-6.064,-13.852,-12.015,-7.938,-19.798,-10.179,-8.803,-11.038,-12.13,-9.999,-17.603,-10.856,-12.074,-5.711,-6.212,-8.929,-8.709,-8.521,-7.65,-15.781,-6.811,-10.427,-14.024,-7.353,-11.492,-4.535,-10.772,-12.833,-9.883,-17.129,-8.834,-9.201,-8.193,-12.467,-10.777,-19.038,-9.245,-9.969,-6.655,-6.64,-10.996,-10.133,-10.815,-6.544,-16.922,-7.101,-11.912,-15.175,-8.12,-11.403,-5.53,-11.1,-12.48,-9.371,-19.263,-9.599,-7.518,-8.427,-13.316,-10.866,-18.482,-10.157,-8.108,-5.683,-6.185,-8.07,-8.174,-10.173,-7.742,-14.069,-7.464,-10.699,-15.421,-8.662,-10.946,-4.788,-11.553,-11.763)

vpcontrol$calc.derivs = TRUE
#> Error in vpcontrol$calc.derivs = TRUE: object 'vpcontrol' not found

formula = y ~ Age + (1 | Individual) + (1 | Tissue)

fit = lmer(formula, info)

set.seed(1)
bootRes = lme4::bootMer( fit, calcVarPart, use.u=TRUE, nsim=100, parallel="no", ncpus=1)
#> Warning in lme4::bootMer(fit, calcVarPart, use.u = TRUE, nsim = 100, parallel =
#> "no", : some bootstrap runs failed (1/100)

bootRes
#> 
#> PARAMETRIC BOOTSTRAP
#> 
#> 
#> Call:
#> lme4::bootMer(x = fit, FUN = calcVarPart, nsim = 100, use.u = TRUE, 
#>     parallel = "no", ncpus = 1)
#> 
#> 
#> Bootstrap Statistics :
#>         original        bias    std. error
#> t1* 8.900659e-01 -0.0040234471 0.021829475
#> t2* 2.665022e-02 -0.0006498789 0.014626225
#> t3* 4.185209e-05  0.0012114981 0.001527238
#> t4* 8.324202e-02  0.0034618279 0.014860092
#> 
#> 1 message(s): boundary (singular) fit: see help('isSingular')
#> 1 error(s): boundary (singular) fit: see help('isSingular')

[bbolker]

I think this is resolved, see lme4/lme4#676 ; still testing though.

[bbolker]

Actually, now I'm not sure. There was a bug that I fixed (bootMer doesn't inherit control settings of the original fit), but if we don't use control to suppress singularity checks then calcVarPart seems to be responsible for escalating messages to errors?

set.seed(1)
bootRes2 = lme4::bootMer( fit, fixef, use.u=TRUE, nsim=100, parallel="no", ncpus=1, verbose=TRUE)

shows a singular-fit message on bootstrap replicate 98.

Running the example above with verbose=TRUE shows that in the same run (98) the message appears as both a message and an error: this must be happening due to calcVarPart ... ? This happens here. If the original model was run with singularity-checking turned off then this doesn't happen, but if it was run "vanilla" (as in this case) then the behaviour that's happening seems to be as intended/coded in calcVarPart ... ?

On the other hand, I can easily imagine that in the less-minimal example you started with, the singularity-checking was being suppressed in the original model fit but then incorrectly ignored by bootMer ... ? Can you re-try your less-minimal example with the most recent GitHub version of lme4 ... ?

[GabrielHoffman]

Thanks for your patience. It seems like the issue was with my internal checks rather than with lme4::bootMer(). Version 1.25.13 posted now includes your original patch and a fix to this latest issue.