lmer() using a lot of cores?

[Ambrosinae]

Hi,

I seem to be having an issue similar to #627, where running one lmer() model in a for loop is using up 70% of my CPU on a 16 core/32 thread 5950x. The model looks something like lmer(Y ~ X1 + X2 + (1 | PatientID)), with each variable having a few hundred values and the random effect having about a hundred levels.

I'm on Windows 10, running R in a Jupyter Notebook.

My session info looks like this with the minimal packages I use loaded:

R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] lubridate_1.9.2 forcats_1.0.0   stringr_1.5.0   dplyr_1.1.1    
 [5] purrr_1.0.1     readr_2.1.4     tidyr_1.3.0     tibble_3.2.1   
 [9] ggplot2_3.4.2   tidyverse_2.0.0 lmerTest_3.1-3  lme4_1.1-32    
[13] Matrix_1.5-4   

loaded via a namespace (and not attached):
 [1] pbdZMQ_0.3-9        tidyselect_1.2.0    repr_1.1.6         
 [4] splines_4.1.3       lattice_0.21-8      colorspace_2.1-0   
 [7] vctrs_0.6.1         generics_0.1.3      htmltools_0.5.5    
[10] base64enc_0.1-3     utf8_1.2.3          rlang_1.1.0        
[13] nloptr_2.0.3        pillar_1.9.0        glue_1.6.2         
[16] withr_2.5.0         uuid_1.1-0          lifecycle_1.0.3    
[19] munsell_0.5.0       gtable_0.3.3        evaluate_0.20      
[22] tzdb_0.3.0          fastmap_1.1.1       fansi_1.0.4        
[25] IRdisplay_1.1       Rcpp_1.0.10         scales_1.2.1       
[28] IRkernel_1.3.2      jsonlite_1.8.4      hms_1.1.3          
[31] digest_0.6.31       stringi_1.7.12      numDeriv_2016.8-1.1
[34] grid_4.1.3          cli_3.6.1           tools_4.1.3        
[37] magrittr_2.0.3      crayon_1.5.2        pkgconfig_2.0.3    
[40] MASS_7.3-58.3       timechange_0.2.0    minqa_1.2.5        
[43] R6_2.5.1            boot_1.3-28.1       nlme_3.1-162       
[46] compiler_4.1.3     

[bbolker]

Were you/are you able to try any of the strategies listed in the linked issue for controlling OpenMP and/or OpenBLAS threading? (I can't see why OpenMP would be involved ...) Does Sys.setenv(OPENBLAS_NUM_THREADS=1) help?

[Ambrosinae]

I did try Sys.setenv(OPENBLAS_NUM_THREADS=4) but there was no difference in CPU utilization.

[bbolker]

Not sure what to do about this (i.e. how to move ahead). Might try to get help from the r-sig-mixed-models@r-project.org mailing list (or r-devel/r-package-devel ...)

[Ambrosinae]

No problem, I'll see what I can do...

[bbolker]

I'm going to close this since there's not really enough information to follow up.