Parallel issues with future

[jayrobwilliams]

I can execute parallel sampling for multiple chains with brm() without issues. However, when I try to use future with kfold() or brm_multiple() I get errors.

library(brms)
library(future)
plan(multiprocess(workers = 4))
bprior1 <- prior(student_t(5,0,10), class = b) +
  prior(cauchy(0,2), class = sd)
# this works
fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient),
            data = epilepsy, family = poisson(), prior = bprior1, future = T)
# this doesn't
kf <- kfold(fit1, chains = 1, K = 5, future = T)
library(mice)
imp <- mice(nhanes2)
# neither does this
fit_imp1 <- brm_multiple(bmi~age+hyp+chl, data = imp, chains = 1, future = T)

The error that I receive from kfold() is:

Fitting model 1 out of 5
Start sampling
failed to create the sampler; sampling not done
Error in rstan::sflist2stanfit(fits) : 
  The following elements of 'sflist' do not contain samples: 1.

The error I receive from brm_multiple() is:

Compiling the C++ model
Start sampling
Error in rstan::sflist2stanfit(fits) : 
  'sflist' should have at least 1 element

I'm using the development version of brms, but I'm attaching the session info to help:

R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] mice_3.3.0      lattice_0.20-38 future_1.12.0   brms_2.7.4      Rcpp_1.0.0     

loaded via a namespace (and not attached):
 [1] nlme_3.1-137         matrixStats_0.54.0   xts_0.10-2           threejs_0.3.1       
 [5] rstan_2.18.2         tools_3.5.2          backports_1.1.3      R6_2.3.0            
 [9] DT_0.4               rpart_4.1-13         lazyeval_0.2.1       colorspace_1.3-2    
[13] nnet_7.3-12          jomo_2.6-2           tidyselect_0.2.4     gridExtra_2.3       
[17] prettyunits_1.0.2    mnormt_1.5-5         processx_3.1.0       Brobdingnag_1.2-6   
[21] compiler_3.5.2       cli_1.0.1            shinyjs_1.0          colourpicker_1.0    
[25] scales_1.0.0         dygraphs_1.1.1.4     mvtnorm_1.0-10       psych_1.8.4         
[29] ggridges_0.5.1       callr_2.0.4          stringr_1.4.0        digest_0.6.18       
[33] StanHeaders_2.18.1   minqa_1.2.4          foreign_0.8-71       base64enc_0.1-3     
[37] pkgconfig_2.0.2      htmltools_0.3.6      lme4_1.1-17          htmlwidgets_1.2     
[41] rlang_0.3.1          rstudioapi_0.9.0     shiny_1.1.0          bindr_0.1.1         
[45] zoo_1.8-1            crosstalk_1.0.0      gtools_3.8.1         dplyr_0.7.6         
[49] inline_0.3.15        magrittr_1.5         loo_2.0.0            bayesplot_1.6.0     
[53] Matrix_1.2-15        munsell_0.5.0        abind_1.4-5          stringi_1.3.1       
[57] debugme_1.1.0        MASS_7.3-51.1        pkgbuild_1.0.2       plyr_1.8.4          
[61] grid_3.5.2           parallel_3.5.2       listenv_0.7.0        promises_1.0.1      
[65] crayon_1.3.4         mitml_0.3-5          miniUI_0.1.1.1       splines_3.5.2       
[69] pillar_1.3.1         igraph_1.2.2         markdown_0.8         shinystan_2.5.0     
[73] reshape2_1.4.3       codetools_0.2-15     stats4_3.5.2         pan_1.4             
[77] rstantools_1.5.1     glue_1.3.0           packrat_0.4.9-3      nloptr_1.0.4        
[81] httpuv_1.4.3         gtable_0.2.0         purrr_0.2.5          tidyr_0.8.1         
[85] assertthat_0.2.0     ggplot2_3.1.0.9000   mime_0.6             xtable_1.8-2        
[89] broom_0.4.4          coda_0.19-2          later_0.7.2          survival_2.43-3     
[93] rsconnect_0.8.8      tibble_2.0.1         shinythemes_1.1.1    bindrcpp_0.2.2      
[97] globals_0.12.4       bridgesampling_0.6-0

I've also noticed that I have to set future = T in each function, unlike the examples in the help files for each function, in case that helps identify the cause of the errors.

[paul-buerkner]

Both kfold and brm_multiple have future automatically activated to run models in parallel, while brm can run chains in parallel if future = TRUE. That is you have two layers of futures on top of each other for the former two functions if you also activate future in brm. I am not sure how future handles this situation exactly, but it is definitely built to deal with it (and it's surely documended somewhere). kfold has future implemented only on the loo-compare branch of brms, which will be merged in a few days. That's why it is not working on the master branch currently.

That said, there was also a bug in how brm runs models if future = TRUE and chains = 0, which is necessary in kfold and brm_multiple. This should be fixed now.

[jayrobwilliams]

Fantastic, thanks for clearing that up. This is my first time using future for parallelization, so I didn't realize that doubling up could cause issues like that. Thanks!