py_module_available("nevergrad") = False #817

Unpickle · 2023-09-15T03:13:36Z

Project Robyn

Describe issue: nevergrad is physically present in "r-reticulate" environment. pip command also shows present. Conda install in R also show requirement satisfied. py(config) is not showing nevergrad. Demo.R is asking for nevergrad.

Question: What else should I do to enable py_config() read nevergrad.

py_module_available("nevergrad"). FALSE
It shows TRUE for all other packages - scipy, pandas.

py_config()
python: C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe
libpython: C:/ProgramData/Miniconda3/envs/r-reticulate/python38.dll
pythonhome: C:/ProgramData/Miniconda3/envs/r-reticulate
version: 3.8.18 (default, Sep 11 2023, 13:39:12) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/ProgramData/Miniconda3/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.24.4

NOTE: Python version was forced by RETICULATE_PYTHON
NOT SHOWING NEVERGRAD

Provide reproducible example

library(reticulate)
reticulate:::rm_all_reticulate_state() #fixed SSL issue
Sys.setenv(RETICULATE_PYTHON = "C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe")
use_python("C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe")
use_condaenv("r-reticulate",required = TRUE)

conda_install("r-reticulate", "nevergrad", pip = TRUE, user = TRUE) ==> Requirement satisfied 0.12.0

I have changed. Renviron
PATH="${RTOOLS42_HOME}\usr\bin;${PATH}"
RETICULATE_PYTHON="C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe"
uncheck the box for "Automatically activate project-local Python environments".

Environment & Robyn version

Make sure you're using the latest Robyn version before you post an issue.

packageVersion("Robyn") [1] ‘3.10.3’
R.version$version.string: R-4.2.3

Unpickle · 2023-09-15T03:31:03Z

I have also forced r-reticulate path in Rstidio GLobal Option Python page.
I am updating R to latest version 4.3.1
I have a window 10 server.
I am specifically requesting laresbernardo and gufengzhou ( I have read your posts on nevergrad installtion) and others for help. It looks like last mile minor issue which I am unable to resolve.

gufengzhou · 2023-09-15T08:51:41Z

This is strange, esp. because you've hardcoded the path in .Renviron. You have installed nevergrad before specifying .Renviron, right? Can you try install it afterwards? Then maybe also restart your machine?

for reference, on my mac, py_config() also doesn't show nevergrad, but py_module_available("nevergrad") is true

Unpickle · 2023-09-15T13:59:02Z

Its indeed strange and disappointing. Was very eager to try Robyn.
I uninstalled nevergrad and ran the following sequence of code. As you can see nevergrad is successfully installed but reticulate fails to recognize it.

library(reticulate)

usethis::edit_r_environ()

reticulate:::rm_all_reticulate_state() #fixed SSL issue
Sys.setenv(RETICULATE_PYTHON = "C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe")
use_python("C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe")
use_condaenv("r-reticulate",required = TRUE)

conda_install("r-reticulate", "nevergrad", pip = TRUE, user = TRUE)

Successfully installed nevergrad-0.12.0
[1] "nevergrad"

py_module_available("nevergrad")
[1] FALSE

gufengzhou · 2023-09-15T15:17:49Z

Very strange. It's kind of known that Sys.setenv() and use_python are not always able to force the desired python path. Are you sure you've put the Renviron file in the right path? It's also not my expertise but I've understood that there could exist multiple Renviron files on either your user home path or the project home path, and R has a hierarchy to read them.

One more idea to do this: you can use the function usethis::edit_r_environ() to edit the Renviron file directly in R. If you check ?edit_r_environ, you can set the scope to user or project to set the Renviron file for each scope. I believe both should work for your purpose, but you can check maybe this to get more context.

And sorry to say this, but you also restarted the R session, right?

Unpickle · 2023-09-15T16:03:06Z

Hi! Thanks for helping out! I followed all the steps - several times. My python and R environ shows nevergrad except py_config(),

Yes - I did restart R session after deleting nevergrad
(r-reticulate) E:\Live-MMM\22.MMM\OpenSource\Robyn\Original_Robyn_Practice>python -m pip show nevergrad
WARNING: Package(s) not found: nevergrad. Restarted R after this step.
I am editing the Renviron directly in R.
RETICULATE_PYTHON="C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe" Retarted after step again.
#Load Libraries#
library(reticulate)
reticulate:::rm_all_reticulate_state() #fixed SSL issue
Sys.setenv(RETICULATE_PYTHON = "C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe")
use_python("C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe")
use_condaenv("r-reticulate",required = TRUE)
conda_install("r-reticulate", "nevergrad", pip = TRUE, user = TRUE)
(r-reticulate) E:\Live-MMM\22.MMM\OpenSource\Robyn\Original_Robyn_Practice>python -m pip show nevergrad
Name: nevergrad
Version: 0.12.0
Installing collected packages: nevergrad
Successfully installed nevergrad-0.12.0
[1] "nevergrad"
nevergrad detected in pyhon as well as R
py_config()
python: C:/ProgramData/Miniconda3/envs/r-reticulate/python.exe
libpython: C:/ProgramData/Miniconda3/envs/r-reticulate/python38.dll
pythonhome: C:/ProgramData/Miniconda3/envs/r-reticulate
version: 3.8.18 (default, Sep 11 2023, 13:39:12) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/ProgramData/Miniconda3/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.24.4

NOTE: Python version was forced by RETICULATE_PYTHON

gufengzhou · 2023-09-15T16:20:32Z

you mentioned Demo.R is asking for nevergrad. What error is there exactly? Can you share a screenshot or so?

Unpickle · 2023-09-15T16:31:31Z

Hi! Thanks for being patient. Robyn is citing lack of nevergrad for error in running
Input data has 208 weeks in total: 2015-11-23 to 2019-11-11
Initial model is built on rolling window of 157 week: 2016-01-04 to 2018-12-31
Time-series validation with train_size range of 50%-80% of the data...
Using geometric adstocking with 20 hyperparameters (20 to iterate + 0 fixed) on 1 core (Windows fallback)

Starting 5 trials with 2000 iterations each using TwoPointsDE nevergrad algorithm...
Running trial 1 of 5
Error in robyn_mmm(InputCollect = InputCollect, hyper_collect = hyper_collect, :
You must have nevergrad python library installed.
Please check our install demo: https://github.com/facebookexperimental/Robyn/blob/main/demo/install_nevergrad.R

gufengzhou · 2023-09-15T16:39:30Z

This error is shown when reticulate::py_module_available("nevergrad") returns FALSE. I'm out of ideas at the moment. Maybe you can ask in the fb group?

Unpickle · 2023-09-15T17:24:40Z

Thanks! Its indeed very strange. I plan to install it locally in a new laptop to test it out.

Tiodocafe · 2023-09-19T10:55:09Z

I am also having this same problem, did you were able to correct this error? My error is the same in every detail that you put over here.

gufengzhou · 2023-09-20T07:01:21Z

reposting here too: one more idea #820 (comment)

andrewrjames · 2023-09-20T07:15:10Z

Hi, I also found similar issue,
I already followed the following code

library(reticulate)
virtualenv_create("r-reticulate")
Sys.setenv(RETICULATE_PYTHON = "~/.virtualenvs/r-reticulate/bin/python")
py_config()

And it shows like this, and when checking nevergrad was never installed

When I tried to install nevergrad, it shows error like this. Can you help me on this?

gufengzhou · 2023-09-20T08:12:06Z

@andrewrjames in your case, you're missing numpy. please try this and let me know if it helps #820 (comment)

andrewrjames · 2023-09-20T10:37:33Z

hi @gufengzhou , I tried to reinstall R and Rstudio and it now works haha.

Tiodocafe · 2023-09-20T12:41:36Z

Yesterday i tried to install Nevergrad in both Windows and MAC PCs that i have here and didn't have succeed, today i reinstall R, RStudio (Tried 3 different Versions, RTools and Python (Tried the Last Version, the 3.8 and the 3.9.12) and still having the same Nevergrad problem =/

Tiodocafe · 2023-09-20T13:43:15Z

Hello guys,
Just made it!!
Instead of install the Nevergrad in Rstudio, i have imputed the code in the R 4.3.1.

After that i just uploaded on Rstudio

Unpickle · 2023-09-20T14:07:53Z

@Tiodocafe Could you please provide more details in how you enabled it. I am using python 3.8 and R studio latest like you.
Thanks in advance

Tiodocafe · 2023-09-20T14:16:14Z

Yes :)

I have deleted my Python and have installed the Python 3.9.12 (https://www.python.org/downloads/release/python-3912/)

After that, i deleted all the documents/files from Reticulate that were in My Documents to start everything since the start.

Instead of use RStudio to configurate Nevergrad, i use the R (The first program that you have installed)

In R, i have done this steps:

Install reticulate first if you haven't already

install.packages("reticulate")

Option 1: nevergrad installation via PIP

1. load reticulate

library("reticulate")

2. create virtual environment

virtualenv_create("r-reticulate")

3. use the environment created

use_virtualenv("r-reticulate", required = TRUE)

4. point Python path to the python file in the virtual environment. Below is

an example for MacOS M1 or above. The "~" is my home dir "/Users/gufengzhou".

Show hidden files in case you want to locate the file yourself.

Sys.setenv(RETICULATE_PYTHON = "~/.virtualenvs/r-reticulate/bin/python")

5. Check python path

py_config() # If the first path is not as 4, do 6

6. Restart R session, run #4 first, then load library("reticulate"), check

py_config() again, python should have path as in #4.

If you see: "NOTE: Python version was forced by RETICULATE_PYTHON_FALLBACK"

if you're using RStudio, go to Global Options > Python, and uncheck the

box for "Automatically activate project-local Python environments".

7. Install numpy if py_config shows it's not available

py_install("numpy", pip = TRUE)

8. Install nevergrad

py_install("nevergrad", pip = TRUE)

After this, i have opened Rstudio and put:

library("reticulate") #To confirm that everthing is alright
py_config() #To understand if everything was alright, still didn't show Nevergrad
py_module_available("nevergrad") #Came with the information that everyhing was alright =)

gufengzhou self-assigned this Sep 15, 2023

gufengzhou added the nevergrad installation label Oct 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

py_module_available("nevergrad") = False #817

py_module_available("nevergrad") = False #817

Unpickle commented Sep 15, 2023

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023 •

edited

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023 •

edited

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023

Unpickle commented Sep 15, 2023

Tiodocafe commented Sep 19, 2023

gufengzhou commented Sep 20, 2023

andrewrjames commented Sep 20, 2023

gufengzhou commented Sep 20, 2023 •

edited

andrewrjames commented Sep 20, 2023

Tiodocafe commented Sep 20, 2023

Tiodocafe commented Sep 20, 2023

Unpickle commented Sep 20, 2023

Tiodocafe commented Sep 20, 2023

py_module_available("nevergrad") = False #817

py_module_available("nevergrad") = False #817

Comments

Unpickle commented Sep 15, 2023

Project Robyn

Describe issue: nevergrad is physically present in "r-reticulate" environment. pip command also shows present. Conda install in R also show requirement satisfied. py(config) is not showing nevergrad. Demo.R is asking for nevergrad.

Provide reproducible example

Environment & Robyn version

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023 • edited

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023 • edited

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023

Unpickle commented Sep 15, 2023

gufengzhou commented Sep 15, 2023

Unpickle commented Sep 15, 2023

Tiodocafe commented Sep 19, 2023

gufengzhou commented Sep 20, 2023

andrewrjames commented Sep 20, 2023

gufengzhou commented Sep 20, 2023 • edited

andrewrjames commented Sep 20, 2023

Tiodocafe commented Sep 20, 2023

Tiodocafe commented Sep 20, 2023

Unpickle commented Sep 20, 2023

Tiodocafe commented Sep 20, 2023

Install reticulate first if you haven't already

Option 1: nevergrad installation via PIP

1. load reticulate

2. create virtual environment

3. use the environment created

4. point Python path to the python file in the virtual environment. Below is

an example for MacOS M1 or above. The "~" is my home dir "/Users/gufengzhou".

Show hidden files in case you want to locate the file yourself.

5. Check python path

6. Restart R session, run #4 first, then load library("reticulate"), check

py_config() again, python should have path as in #4.

If you see: "NOTE: Python version was forced by RETICULATE_PYTHON_FALLBACK"

if you're using RStudio, go to Global Options > Python, and uncheck the

box for "Automatically activate project-local Python environments".

7. Install numpy if py_config shows it's not available

8. Install nevergrad

gufengzhou commented Sep 15, 2023 •

edited

gufengzhou commented Sep 15, 2023 •

edited

gufengzhou commented Sep 20, 2023 •

edited