Error on install_keras(); virtualenv is not supported on windows

[venciso]

Hi, I installed Keras. When I do install_keras() I get this error:

Error: Installing Python packages into a virtualenv is not supported on Windows
I get the same even if I do

install_keras(method="conda") or install_keras(method="auto")

This is the output of py_config() if it helps:

python:         C:/ProgramData/Anaconda3/python.exe
libpython:      C:/ProgramData/Anaconda3/python36.dll
pythonhome:     C:/ProgramData/Anaconda3
version:        3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/ProgramData/Anaconda3/Lib/site-packages/numpy
numpy_version:  1.18.1
tensorflow:     C:\PROGRA~3\ANACON~1\lib\site-packages\tensorflow\__init__.p

I'm using R 4.0.0. This is sessionInfo():

R version 4.0.0 (2020-04-24)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252    LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                            LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.16 keras_2.3.0.0  

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4.6       lubridate_1.7.8    lattice_0.20-41    tidyr_1.1.0        listenv_0.8.0      class_7.3-16       assertthat_0.2.1  
 [8] zeallot_0.1.0      digest_0.6.25      packrat_0.5.0      ipred_0.9-9        foreach_1.5.0      R6_2.4.1           plyr_1.8.6        
[15] tune_0.1.0         ggplot2_3.3.1      pillar_1.4.4       tfruns_1.4         rlang_0.4.6        parsnip_0.1.1      rstudioapi_0.11   
[22] whisker_0.4        DiceDesign_1.8-1   furrr_0.1.0        rpart_4.1-15       Matrix_1.2-18      splines_4.0.0      gower_0.2.1       
[29] munsell_0.5.0      compiler_4.0.0     xfun_0.14          base64enc_0.1-3    pkgconfig_2.0.3    tensorflow_2.2.0   globals_0.12.5    
[36] dials_0.0.6        nnet_7.3-13        tidyselect_1.1.0   tibble_3.0.1       prodlim_2019.11.13 codetools_0.2-16   workflows_0.1.1   
[43] GPfit_1.0-8        fansi_0.4.1        future_1.17.0      crayon_1.3.4       dplyr_1.0.0        withr_2.2.0        rappdirs_0.3.1    
[50] MASS_7.3-51.5      recipes_0.1.12     grid_4.0.0         jsonlite_1.6.1     gtable_0.3.0       lifecycle_0.2.0    magrittr_1.5      
[57] pROC_1.16.2        scales_1.1.1       cli_2.0.2          rsample_0.0.7      timeDate_3043.102  ellipsis_0.3.1     lhs_1.0.2         
[64] generics_0.0.2     vctrs_0.3.0        lava_1.6.7         iterators_1.0.12   yardstick_0.0.6    tools_4.0.0        glue_1.4.1        
[71] purrr_0.3.4        parallel_4.0.0     survival_3.1-12    yaml_2.2.1         colorspace_1.4-1   knitr_1.28

Also, I can load both TF and Keras in Python.

Any help would be greatly appreciated.

Thanks.

[turgut090]

Please, refer to: https://rstudio.github.io/reticulate/articles/python_packages.html

conda_create("r-reticulate")       
use_condaenv("r-reticulate")   
conda_install("r-reticulate",c('keras','tensorflow'), pip = TRUE)

[venciso]

Hi henry090 thank you for you reply.
I ran the code and it completed without errors.

I then tried:
mnist <- dataset_mnist()

and got the following error.

Error: Installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 C:\Users\part of path with spaces\Documents\.conda\envs\r-reticulate\python.exe
 C:\ProgramData\Anaconda3\python.exe
 C:\Users\part of path with spaces\Documents\.conda\envs\spacy_condaenv\python.exe

You can install TensorFlow using the install_tensorflow() function.

I closed my RStudio session and opened a new one and ran the following

> library(keras)
> use_condaenv("r-reticulate") 
> install_keras()

and got:

Error: installation of 'python=3.6' into environment 'C:/Users/part of path with spaces/Documents/.conda/envs/r-reticulate' failed [error code 127]

Tried getting mnist again:
> mnist <- dataset_mnist()
Same message. I don't really understand why it cannot find tensorflow, it's looking in the env I created.

Error: Installation of TensorFlow not found.

Python environments searched for 'tensorflow' package:
 C:\Users\part of path with spaces\Documents\.conda\envs\r-reticulate\python.exe
 C:\ProgramData\Anaconda3\python.exe
 C:\Users\part of path with spaces\Documents\.conda\envs\spacy_condaenv\python.exe

You can install TensorFlow using the install_tensorflow() function.

Please note that I'm censoring part of my path for privacy reasons but I want to show that it's got spaces in it in case this could be part of the problem.

Something else I don't understand is if I have to run install_keras() every time I open RStudio or if require(keras) should suffice because I'm calling use_condaenv.

Thanks

[turgut090]

Does this help from fresh Session:

reticulate::use_python('C:/Users/part of path with spaces/Documents/.conda/envs/r-reticulate/python.exe', required = TRUE)
tensorflow::install_tensorflow()

[venciso]

Hi henry090.

No. Now I get this:

 tensorflow::install_tensorflow()
Error: installation of 'python=3.6' into environment 'C:/Users/part of path with spaces/Documents/.conda/envs/r-reticulate' failed [error code 127]

[turgut090]

Try again to install like this:

use_condaenv("r-reticulate")   
conda_install("r-reticulate",c('keras','tensorflow'), pip = TRUE)

Then specify python path:

reticulate::use_python('C:/Users/part of path with spaces/Documents/.conda/envs/r-reticulate/python.exe', required = TRUE)

[venciso]

Hi henry09,
I get the same error again for both tensorflow and keras.
Any further ideas would be greatly appreciated!

use_condaenv("r-reticulate")   
conda_install("r-reticulate",c('keras','tensorflow'), pip = TRUE)
reticulate::use_python('C:/Users/part of path with spaces/Documents/.conda/envs/r-reticulate/python.exe', required = TRUE)
require(tensorflow)
install_tensorflow()

Error: installation of 'python=3.6' into environment 'C:/Users/part of path with spaces/Documents/.conda/envs/r-reticulate' failed [error code 127]

[turgut090]

Please, do not use install_tensorflow(). Install with conda_install("r-reticulate",c('keras','tensorflow'), pip = TRUE). Then use reticulate::use_python('C:/Users/victor.enciso/../Documents/.conda/envs/r-reticulate/python.exe', required = TRUE). Finally, reticulate::py_module_available('tensorflow'). If TRUE then it is already installed.

[venciso]

As for your new suggestion, I do all the steps as you say but I get FALSE.

reticulate::py_module_available('tensorflow')
[1] FALSE

Even though everything is already installed:

> conda_install("r-reticulate",c('keras','tensorflow'), pip = TRUE)
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

I get the same also after I delete the environment and try the process from scratch.

[snoopyhaimaxuan]

Hi I got into the same problem. Have you found a solution?

[nagyz222]

1. I did it my way. I created a conda environment, only with the numpy and keras packages. It will work only for R.
   (Python command line)>

   conda create -n R36 python=3.6 numpy keras requests Pillow scipy
   conda activate R36
   conda install -c conda-forge hdf5=1.10.5 pyyaml==3.12
   pip install --force-reinstall --ignore-installed https://files.pythonhosted.org/packages/72/b8/2ef7057c956f1062ffab750a90a6bdcd3de127fb696fb64583c2dfe77aab/tensorflow-2.2.0-cp36-cp36m-win_amd64.whl

source: https://pypi.org/project/tensorflow/#files

2. In R.4.0.0
   deleted the previous keras/tensorflow (in case of error, delete lib directories)

   remove.packages("keras")
   remove.packages("tensorflow")

defined to reticulate Python env in R

reticulate::use_python("C:/..../Anaconda3/envs/R36/python.exe", required = TRUE)
reticulate::use_condaenv("R36")
reticulate::py_config() 

it must be something similar result:
python: C:/Anaconda3/envs/R36/python.exe
libpython: C:/Anaconda3/envs/R36/python36.dll
pythonhome: C:/Anaconda3/envs/R36
version: 3.6.10 |Anaconda, Inc.| (default, May 7 2020, 19:46:08) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Anaconda3/envs/R36/Lib/site-packages/numpy
numpy_version: 1.18.1

Python and numpy package in the appropriate envelope is now linked to R.

installed from github/source install

reticulate::use_python("C:/..../Anaconda3/envs/R36/python.exe", required = TRUE)
reticulate::use_condaenv("R36")
devtools::install_github("rstudio/keras", type = "source", dependencies = TRUE)
devtools::install_github("rstudio/tensorflow", type = "source", dependencies = TRUE)

restarted R

reticulate::use_python("C:/..../Anaconda3/envs/R36/python.exe", required = TRUE)
reticulate::use_condaenv("R36")
keras::install_keras()    

R starts to collect, download and install all packages if needed. Watch the error messages and set PATH in case of message about it.

3. Reboot
   Created in your R default working directory .Rprofile text file with Notepad or similar, no extension in the file name).
   Content is one line:

   Sys.setenv(RETICULATE_PYTHON = "C:/..../Anaconda3/envs/R36/python.exe")

Started R and specified the Python env.

reticulate::use_python("C:/..../Anaconda3/envs/R36/python.exe", required = TRUE)
reticulate::use_condaenv("R36")
library("keras")
library("tensorflow")

[janak12jp]

Try the following method which helped for me on Windows 10 machine:
1 Uninstall the packages tensorflow, keras, tfruns
2 If you haven't installed anaconda, install it. If you have already, update it using conda update --all
3 Also install latest rtools based on your R vesion
4 Open Rstudio and run the following commands

install.packages("reticulate")
install.packages("remotes")
library(reticulate)
library(remotes)
install_github("rstudio/tensorflow")
library(tensorflow)
install_tensorflow(version = "1.15.0-gpu", method = "conda", envname = "r-tf")

install_github("rstudio/keras", dependencies = TRUE)
library(keras)`

The above code installs Keras as a tensorflow-backend
Now you can check loading mnist=dataset_mnist() in RStudio.
Also, in python from r-tf environment, try running from tensorflow import keras.

[ThomasWolf0701]

This seems to be a general bug, as I also get the same error since using R4.03 Has this been recognized as an official bug ?

This is the output of py_config() if it helps:
python: C:/Users/Thomas Wolf/anaconda3/python.exe
libpython: C:/Users/Thomas Wolf/anaconda3/python37.dll
pythonhome: C:/Users/Thomas Wolf/anaconda3
version: 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/Thomas Wolf/anaconda3/Lib/site-packages/numpy
numpy_version: 1.18.1
tensorflow: C:\Users\THOMAS1\ANACON1\lib\site-packages\tensorflow_init_.p

python versions found:
C:/Users/Thomas Wolf/anaconda3/python.exe
C:/Users/Thomas Wolf/anaconda3/envs/my-rdkit-env/python.exe

And session info:
R version 4.0.3 (2020-10-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19041)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252 LC_MONETARY=German_Germany.1252 LC_NUMERIC=C
[5] LC_TIME=German_Germany.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] reticulate_1.16 keras_2.3.0.0

loaded via a namespace (and not attached):
[1] Rcpp_1.0.5 lattice_0.20-41 zeallot_0.1.0 rappdirs_0.3.1 grid_4.0.3 R6_2.4.1 jsonlite_1.7.1 magrittr_1.5
[9] tfruns_1.4 whisker_0.4 Matrix_1.2-18 generics_0.0.2 tools_4.0.3 compiler_4.0.3 base64enc_0.1-3 tensorflow_2.2.0

[ThomasWolf0701]

install_github("rstudio/keras", dependencies = TRUE)
library(keras)`

This worked for me.

[PeerW]

@nagyz222
This just saved my day. I have tried to solve this for hours.
Thanks for the thorough guidelines!!

[t-kalinowski]

Hello,

The code that installs tensorflow and keras has been refactored.
Please update to the latest versions of reticulate, tensorflow, and keras:

install.packages("remotes")
remotes::install_github(paste0("rstudio/", c("reticulate", "tensorflow", "keras")))
reticulate::install_miniconda() # skip this if you want to self-install conda or use venv
keras::install_keras()

Please let us know if after updating you are still encountering installation issues:
https://github.com/rstudio/tensorflow/issues/new