TF2 - Failure (test_inference.R:375): mcmc works in parallel - cannot find TF installation when using future

[njtierney]

expr threw an unexpected error. Message: Valid installation of TensorFlow not found.

For some reason setting a future plan makes it not work anymore
We initially added a note in
9f33fe0
with a stop command stating,

"# stop("hi...from the future")"

So perhaps search for that to identify where the bug is ocurring.

Here's the code that fails:

devtools::load_all(".")
#> ℹ Loading greta
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> 
#> ✔ Initialising python and checking dependencies ... done!
#> Loaded Tensorflow version 2.10.0
source(here::here("tests", "testthat", "helpers.R"))
m <- model(normal(0, 1))

op <- future::plan()
# put the future plan back as we found it
withr::defer(future::plan(op))
future::plan(future::multisession)

# one chain
expect_ok(draws <- mcmc(m,
                        warmup = 10, n_samples = 10,
                        chains = 1,
                        verbose = FALSE
))
#> Error: `expr` threw an unexpected error.
#> Message: Valid installation of TensorFlow not found.
#> 
#> Python environments searched for 'tensorflow' package:
#>  /usr/bin/python3
#> 
#> Python exception encountered:
#>  Traceback (most recent call last):
#>   File "/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/reticulate/python/rpytools/loader.py", line 39, in _import_hook
#>     module = _import(
#> ModuleNotFoundError: No module named 'tensorflow'
#> 
#> 
#> You can install TensorFlow using the install_tensorflow() function.
#> 
#> Class:   simpleError/error/condition

^{Created on 2022-12-09 with reprex v2.0.2}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.2.1 (2022-06-23)
#>  os       macOS Monterey 12.3.1
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    en_US.UTF-8
#>  tz       Australia/Perth
#>  date     2022-12-09
#>  pandoc   2.19.2 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  !  package     * version    date (UTC) lib source
#>     abind         1.4-5      2016-07-21 [1] CRAN (R 4.2.0)
#>     backports     1.4.1      2021-12-13 [1] CRAN (R 4.2.0)
#>     base64enc     0.1-3      2015-07-28 [1] CRAN (R 4.2.0)
#>     brio          1.1.3      2021-11-30 [1] CRAN (R 4.2.0)
#>     cachem        1.0.6      2021-08-19 [1] CRAN (R 4.2.0)
#>     callr         3.7.3      2022-11-02 [1] CRAN (R 4.2.0)
#>     cli           3.4.1      2022-09-23 [1] CRAN (R 4.2.0)
#>     coda          0.19-4     2020-09-30 [1] CRAN (R 4.2.0)
#>     codetools     0.2-18     2020-11-04 [1] CRAN (R 4.2.1)
#>     crayon        1.5.2      2022-09-29 [1] CRAN (R 4.2.0)
#>     desc          1.4.2      2022-09-08 [1] CRAN (R 4.2.0)
#>     devtools      2.4.5      2022-10-11 [1] CRAN (R 4.2.0)
#>     digest        0.6.30     2022-10-18 [1] CRAN (R 4.2.0)
#>     ellipsis      0.3.2      2021-04-29 [1] CRAN (R 4.2.0)
#>     evaluate      0.18       2022-11-07 [1] CRAN (R 4.2.0)
#>     fansi         1.0.3      2022-03-24 [1] CRAN (R 4.2.0)
#>     fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.2.0)
#>     fs            1.5.2      2021-12-08 [1] CRAN (R 4.2.0)
#>     future        1.29.0     2022-11-06 [1] CRAN (R 4.2.0)
#>     globals       0.16.2     2022-11-21 [1] CRAN (R 4.2.1)
#>     glue          1.6.2      2022-02-24 [1] CRAN (R 4.2.0)
#>  VP greta       * 0.4.2.9000 2022-09-08 [?] CRAN (R 4.2.0) (on disk 0.4.3)
#>     here          1.0.1      2020-12-13 [1] CRAN (R 4.2.0)
#>     highr         0.9        2021-04-16 [1] CRAN (R 4.2.0)
#>     hms           1.1.2      2022-08-19 [1] CRAN (R 4.2.0)
#>     htmltools     0.5.3      2022-07-18 [1] CRAN (R 4.2.0)
#>     htmlwidgets   1.5.4      2021-09-08 [1] CRAN (R 4.2.0)
#>     httpuv        1.6.6      2022-09-08 [1] CRAN (R 4.2.0)
#>     jsonlite      1.8.3      2022-10-21 [1] CRAN (R 4.2.0)
#>     knitr         1.41       2022-11-18 [1] CRAN (R 4.2.0)
#>     later         1.3.0      2021-08-18 [1] CRAN (R 4.2.0)
#>     lattice       0.20-45    2021-09-22 [1] CRAN (R 4.2.1)
#>     lifecycle     1.0.3      2022-10-07 [1] CRAN (R 4.2.0)
#>     listenv       0.8.0      2019-12-05 [1] CRAN (R 4.2.0)
#>     magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.2.0)
#>     Matrix        1.5-3      2022-11-11 [1] CRAN (R 4.2.0)
#>     memoise       2.0.1      2021-11-26 [1] CRAN (R 4.2.0)
#>     mime          0.12       2021-09-28 [1] CRAN (R 4.2.0)
#>     miniUI        0.1.1.1    2018-05-18 [1] CRAN (R 4.2.0)
#>     parallelly    1.32.1     2022-07-21 [1] CRAN (R 4.2.0)
#>     pillar        1.8.1      2022-08-19 [1] CRAN (R 4.2.0)
#>     pkgbuild      1.4.0      2022-11-27 [1] CRAN (R 4.2.1)
#>     pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.2.0)
#>     pkgload       1.3.2      2022-11-16 [1] CRAN (R 4.2.0)
#>     png           0.1-8      2022-11-29 [1] CRAN (R 4.2.0)
#>     prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.2.0)
#>     processx      3.8.0      2022-10-26 [1] CRAN (R 4.2.0)
#>     profvis       0.3.7      2020-11-02 [1] CRAN (R 4.2.0)
#>     progress      1.2.2      2019-05-16 [1] CRAN (R 4.2.0)
#>     promises      1.2.0.1    2021-02-11 [1] CRAN (R 4.2.0)
#>     ps            1.7.2      2022-10-26 [1] CRAN (R 4.2.0)
#>     purrr         0.3.5      2022-10-06 [1] CRAN (R 4.2.0)
#>     R.cache       0.16.0     2022-07-21 [1] CRAN (R 4.2.0)
#>     R.methodsS3   1.8.2      2022-06-13 [1] CRAN (R 4.2.0)
#>     R.oo          1.25.0     2022-06-12 [1] CRAN (R 4.2.0)
#>     R.utils       2.12.2     2022-11-11 [1] CRAN (R 4.2.0)
#>     R6            2.5.1      2021-08-19 [1] CRAN (R 4.2.0)
#>     Rcpp          1.0.9      2022-07-08 [1] CRAN (R 4.2.0)
#>     remotes       2.4.2      2021-11-30 [1] CRAN (R 4.2.0)
#>     reprex        2.0.2      2022-08-17 [1] CRAN (R 4.2.0)
#>     reticulate    1.26       2022-08-31 [1] CRAN (R 4.2.0)
#>     rlang         1.0.6      2022-09-24 [1] CRAN (R 4.2.0)
#>     rmarkdown     2.18       2022-11-09 [1] CRAN (R 4.2.0)
#>     rprojroot     2.0.3      2022-04-02 [1] CRAN (R 4.2.0)
#>     rstudioapi    0.14       2022-08-22 [1] CRAN (R 4.2.0)
#>     sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.2.0)
#>     shiny         1.7.3      2022-10-25 [1] CRAN (R 4.2.0)
#>     stringi       1.7.8      2022-07-11 [1] CRAN (R 4.2.0)
#>     stringr       1.5.0      2022-12-02 [1] CRAN (R 4.2.0)
#>     styler        1.8.1      2022-11-07 [1] CRAN (R 4.2.0)
#>     tensorflow    2.9.0      2022-05-21 [1] CRAN (R 4.2.0)
#>     testthat    * 3.1.5      2022-10-08 [1] CRAN (R 4.2.0)
#>     tfautograph   0.3.2      2021-09-17 [1] CRAN (R 4.2.0)
#>     tfruns        1.5.1      2022-09-05 [1] CRAN (R 4.2.0)
#>     urlchecker    1.0.1      2021-11-30 [1] CRAN (R 4.2.0)
#>     usethis       2.1.6      2022-05-25 [1] CRAN (R 4.2.0)
#>     utf8          1.2.2      2021-07-24 [1] CRAN (R 4.2.0)
#>     vctrs         0.5.1      2022-11-16 [1] CRAN (R 4.2.0)
#>     whisker       0.4        2019-08-28 [1] CRAN (R 4.2.0)
#>     withr         2.5.0      2022-03-03 [1] CRAN (R 4.2.0)
#>     xfun          0.35       2022-11-16 [1] CRAN (R 4.2.0)
#>     xtable        1.8-4      2019-04-21 [1] CRAN (R 4.2.0)
#>     yaml          2.3.6      2022-10-18 [1] CRAN (R 4.2.0)
#>     yesno         0.1.2      2020-07-10 [1] CRAN (R 4.2.0)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library
#> 
#>  V ── Loaded and on-disk version mismatch.
#>  P ── Loaded and on-disk path mismatch.
#> 
#> ─ Python configuration ───────────────────────────────────────────────────────
#>  python:         /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/bin/python
#>  libpython:      /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/libpython3.8.dylib
#>  pythonhome:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2:/Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2
#>  version:        3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:05:16)  [Clang 12.0.1 ]
#>  numpy:          /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.8/site-packages/numpy
#>  numpy_version:  1.23.2
#>  tensorflow:     /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.8/site-packages/tensorflow
#>  
#>  NOTE: Python version was forced by use_python function
#> 
#> ──────────────────────────────────────────────────────────────────────────────