Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Isolate conda R from system user library. #65

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

jdblischak
Copy link
Member

Checklist

  • Used a fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

It's a constant frustration for me that conda R uses the R packages installed in the R user library. This is also a major limitation when trying to convince new conda users about the reproducibility benefits of conda.

This PR attempts to do the following:

  • Disable the default setting of R_LIBS_USER by r-base
  • Temporarily unsets any custom R_LIBS_USER set by the user
  • Tests the isolation during package creation by creating an empty user library that matches the pattern that would be used on that OS

Note to future self: I created the patch by 1) cloning r-source, 2) editing and committing etc/Renviron.in, and 3) running git format-patch HEAD~1.

This issue has previously been discussed in #37.

cc: @mingwandroid @ebolyen @bgruening @isuruf

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@mingwandroid
Copy link
Contributor

Thanks so much for doing this @jdblischak and sorry for the slow review.

Do you (or any of @conda-forge/r) think we should add an env var around all of this as an escape hatch to bring back the old behaviour?

@dpryan79
Copy link
Contributor

dpryan79 commented Jan 7, 2019

An env solution to invoke the old behavior would be welcome, but not a deal breaker (I've lost track of the number of people who have been bitten by libraries in their home directories).

@jdblischak
Copy link
Member Author

Do you (or any of @conda-forge/r) think we should add an env var around all of this as an escape hatch to bring back the old behaviour?

@mingwandroid I think that's fine. However, I think it would only work if the user has explicitly set R_LIBS_USER. The default search location for a local R library is disabled by the patch during the building of the conda package.

A few follow-up questions:

  1. Could you share an example of a recipe that modifies it's behavior based on an environment variable that can be set by the user?
  2. For consistency, do you plan to implement the same behavior for r-base in defaults?
  3. Where should this be documented so that conda R users know about this behavior?

And a general question, what is the purpose of the subsection files: for tests:? I thought it would execute these files, but that only works if they are put in the commands: section. Does this remove this from the final recipe or something like that?

@mingwandroid
Copy link
Contributor

@mingwandroid I think that's fine. However, I think it would only work if the user has explicitly set R_LIBS_USER. The default search location for a local R library is disabled by the patch during the building of the conda package.

Yeah, I propose adding env() checks to your patch. I'll try to schedule some time when I next update default's R to build on top of this PR.

For consistency, do you plan to implement the same behavior for r-base in defaults?

We do not want any difference between R in conda-forge and the R in defaults.

Where should this be documented so that conda R users know about this behavior?

Not sure! I guess conda-forge should document it and we should also document it.

And a general question, what is the purpose of the subsection files: for tests:?

This specifies which files (from the recipe, and not from ${SRC_DIR}, those are specified via source_files:) need to be copied into the test folder / CWD (this is not the test env.) during tests. Packages can be independently tested so we need to know exactly which files are necessary for the tests to work.

@jdblischak
Copy link
Member Author

Yeah, I propose adding env() checks to your patch. I'll try to schedule some time when I next update default's R to build on top of this PR.

@mingwandroid That'd be great. I'm not familiar with env() checks.

We do not want any difference between R in conda-forge and the R in defaults.

100% agree.

Not sure! I guess conda-forge should document it and we should also document it.

Yeah, that is an Issue. Especially for conda-forge since its docs are about the channel, not how to use any given package that it distributes. The Anaconda docs has a section on using R, so that could be a potential options.

https://docs.anaconda.com/anaconda/user-guide/tasks/use-r-language/

This specifies which files (from the recipe, and not from ${SRC_DIR}, those are specified via source_files:) need to be copied into the test folder / CWD (this is not the test env.) during tests. Packages can be independently tested so we need to know exactly which files are necessary for the tests to work.

OK. Makes sense. Thanks for the explanation!

@jdblischak
Copy link
Member Author

@mingwandroid I was just reminded about this PR from a colleague whose code broke because the R packages were being loaded from his user library instead of the conda environment. I think this isolation is very important for conda R users. If the env() checks are documented somewhere, could you please send me a link so that I can try to implement it?

@jdblischak
Copy link
Member Author

I've added support for an environment variable CONDA_KEEP_R_LIBS_USER (name suggestions are welcome). If the user has this defined to anything, then the value of R_LIBS_USER will not be erased when the conda environment is activated.

@jdblischak
Copy link
Member Author

@halldc I am updating activate-r-base.sh and am using the same strategy of checking environment variables that you used in PR #59. Could you please explain the function of the +x that you included in the variable name? I wasn't able to find any documentation on this.

if [[ ! -z ${RSTUDIO_WHICH_R+x} ]]; then

@jdblischak
Copy link
Member Author

@msarahan @jjhelmus @katietz Could you please review this PR to r-base? I am attempting to isolate R installed in a conda environment from the user's local R library (which always takes precedence when searching for R packages). Any feedback would be much appreciated.

If you approve of the approach, I will start working on support for Windows. Thanks!

@mingwandroid
Copy link
Contributor

mingwandroid commented May 28, 2019

Sounds fine to me. Sorry not a detailed review in the least! Pinging @katietz

@halldc
Copy link
Contributor

halldc commented May 28, 2019

@jdblischak This is used to evaluate whether the parameter is set. ${var+x} is a parameter expansion which evaluates to nothing if var is unset, and substitutes the string x otherwise. See this Stackoverflow post for details.

@jdblischak
Copy link
Member Author

${var+x} is a parameter expansion which evaluates to nothing if var is unset, and substitutes the string x otherwise.

@halldc Thanks for the explanation and link. The key for me from that SO answer was that using +x enables the script to "distinguish between a variable that is unset and a variable that is set to the empty string."

@jdblischak
Copy link
Member Author

Just got the email from the R Project: "R 3.6.1 scheduled for July 5". It would be nice if we could get this feature into the conda-forge release of r-base 3.6.1.

@jdblischak
Copy link
Member Author

For any users that find this thread and need a short-term solution, @dpryan79 provided a good solution in conda-forge/r-rjags-feedstock#6 (comment):

.libPaths(R.home("library"))

This will replace your default user library with the system library path (which is where conda R packages are installed). To automate this, you could put it inside of an .Rprofile file inside your project directory (you wouldn't want to put it in your home directory because then this would always ignore your user library, which you may want to use for other projects).

@h-vetinari
Copy link
Member

Can this be retried now that 3.6.1 is out?

Copy link

@katietz katietz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, that review took a bit. Change looks sensible to me.
As 3.6.1 is out now, is that patch applying to new version too?
Btw in my opionion we should update soon to 3.6.1 version

@jdblischak
Copy link
Member Author

Can this be retried now that 3.6.1 is out?

@h-vetinari I rebased the PR against R 3.6.1. Could you please test it out?

@jdblischak
Copy link
Member Author

Sorry, that review took a bit. Change looks sensible to me.

@katietz Thanks for the review! Can I interpret this as you being open to the same patch to the r-base recipe in the defaults channel? I'd prefer if the two behaved the same in regards to the user library.

@h-vetinari
Copy link
Member

@jdblischak: @h-vetinari I rebased the PR against R 3.6.1. Could you please test it out?

I can build on my personal laptop, but in the environment where I actually need it, I can't pull stuff from github (but we have a proxy for conda-forge)... There's some complicated ways around it, but even then I'd have to wait until conda-forge supports an r=3.6.1 install. That migration is in progress, at least...

@katietz
Copy link

katietz commented Jul 18, 2019

Yes. I am openminded to put patch also in r-base default channel for 3.6.1

@dpryan79
Copy link
Contributor

@h-vetinari The migration is effectively finished if you can try things out.

@h-vetinari
Copy link
Member

@dpryan79
Thanks for all the rebuilding effort!

In a fresh conda install, doing

conda config --add channels conda-forge \
    && conda update -n base conda \
    && conda update --all \
    && conda install --prune --strict-channel-priority R=3.6

fails with

[...]
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with current_repodata.json, will retry with next repodata source.
Initial quick solve with frozen env failed.  Unfreezing env and trying again.
Solving environment: ...working... failed with current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed
Initial quick solve with frozen env failed.  Unfreezing env and trying again.
Solving environment: ...working... failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:
Package tk conflicts for:
python=3.7 -> tk[version='>=8.6.7,<8.7.0a0|>=8.6.8,<8.7.0a0|>=8.6.9,<8.7.0a0']
Package libffi conflicts for:
python=3.7 -> libffi[version='>=3.2.1,<3.3.0a0|>=3.2.1,<4.0a0']
Package zlib conflicts for:
python=3.7 -> zlib[version='>=1.2.11,<1.3.0a0']
Package libstdcxx-ng conflicts for:
python=3.7 -> libstdcxx-ng[version='>=4.9|>=7.2.0|>=7.3.0']
Package sqlite conflicts for:
python=3.7 -> sqlite[version='>=3.24.0,<4.0a0|>=3.25.1,<4.0a0|>=3.25.2,<4.0a0|>=3.25.3,<4.0a0|>=3.26.0,<4.0a0|>=3.27.2,<4.0a0|>=3.28.0,<4.0a0']
Package xz conflicts for:
python=3.7 -> xz[version='>=5.2.3,<5.3.0a0|>=5.2.4,<5.3.0a0|>=5.2.4,<6.0a0']
Package libgcc-ng conflicts for:
python=3.7 -> libgcc-ng[version='>=4.9|>=7.2.0|>=7.3.0']
Package bzip2 conflicts for:
python=3.7 -> bzip2[version='>=1.0.6,<2.0a0']
Package openssl conflicts for:
python=3.7 -> openssl[version='>=1.0.2o,<1.0.3a|>=1.0.2p,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1b,<1.1.2a']
Package readline conflicts for:
python=3.7 -> readline[version='>=7.0,<8.0a0|>=8.0,<9.0a0']
Package ncurses conflicts for:
python=3.7 -> ncurses[version='>=6.1,<6.2.0a0|>=6.1,<7.0a0']
Package pip conflicts for:
python=3.7 -> pip
Note that strict channel priority may have removed packages required for satisfiability.

Removing --strict-channel-priority runs through, but interestingly, it does not pick up r-recommended or r from conda-forge, even though those have been rebuilt already:

  [...]
  pthread-stubs      conda-forge/linux-64::pthread-stubs-0.4-h14c3975_1001
  r                  pkgs/r/linux-64::r-3.6.0-r36_0
  r-base             conda-forge/linux-64::r-base-3.6.1-h6e652e1_3
  r-boot             conda-forge/noarch::r-boot-1.3_23-r36h6115d3f_2
  r-class            conda-forge/linux-64::r-class-7.3_15-r36hcdcec82_1001
  r-cluster          conda-forge/linux-64::r-cluster-2.1.0-r36h9bbef5b_1
  r-codetools        conda-forge/noarch::r-codetools-0.2_16-r36h6115d3f_1001
  r-foreign          conda-forge/linux-64::r-foreign-0.8_71-r36hcdcec82_1003
  r-kernsmooth       conda-forge/linux-64::r-kernsmooth-2.23_15-r36h9bbef5b_1003
  r-lattice          conda-forge/linux-64::r-lattice-0.20_38-r36hcdcec82_1001
  r-mass             conda-forge/linux-64::r-mass-7.3_51.4-r36hcdcec82_1
  r-matrix           conda-forge/linux-64::r-matrix-1.2_17-r36hcdcec82_1
  r-mgcv             conda-forge/linux-64::r-mgcv-1.8_28-r36hcdcec82_1
  r-nlme             conda-forge/linux-64::r-nlme-3.1_140-r36h9bbef5b_1
  r-nnet             conda-forge/linux-64::r-nnet-7.3_12-r36hcdcec82_1003
  r-recommended      pkgs/r/linux-64::r-recommended-3.6.0-r36_0
  r-rpart            conda-forge/linux-64::r-rpart-4.1_15-r36hcdcec82_1
  r-spatial          conda-forge/linux-64::r-spatial-7.3_11-r36hcdcec82_1003
  r-survival         conda-forge/linux-64::r-survival-2.44_1.1-r36hcdcec82_1
  sed                conda-forge/linux-64::sed-4.7-h1bed415_1000
  [...]

@h-vetinari
Copy link
Member

@dpryan79
I should have been more explicit in my answer: I can build this PR on linux, but I couldn't test it beyond that* until I can get a standard R-install from conda-forge. I opened conda-forge/r-feedstock#12 to track this.

*Or so I thought; actually, all the packages that make up r can already be installed directly. How did you envision that I test this PR?

@jdblischak
Copy link
Member Author

I also used the same strategy to handle the environment variable R_LIBS, which is the issue in #129

@isuruf
Copy link
Member

isuruf commented May 13, 2020

-1 to this. I think we should only warn that R_LIBS and R_LIBS_USER is set in the activation script. It's not good to deviate from upstream behaviour like this.

@jdblischak
Copy link
Member Author

It's not good to deviate from upstream behaviour like this.

@isuruf Thanks for the feedback. Could you please elaborate on what you mean by "upstream behaviour"? Are you referring to r-base in the defaults channel or R itself?

@isuruf
Copy link
Member

isuruf commented May 13, 2020

I meant R itself.

@jdblischak
Copy link
Member Author

I'm not sure I understand then. The environment variables like R_LIBS_USER are for R installations that distinguish between system packages (shared by all users on the machine) and user packages. R installed via conda already breaks this paradigm because it is a local installation for only one user.

As an R user, the packages I have installed in my user directory are for use with the R installed on my system. When I activate a conda environment that has R installed, it is because I am specifically trying to isolate my analysis from whatever I happened to have installed on that computer (which likely differs from my other computers, and definitely differs from the computers of my collaborators). If the activate script only warns about the presence of a user directory, then that leaves me to try to come up with some way to get conda R to ignore it (and then I have to help my collaborators do the same).

@isuruf
Copy link
Member

isuruf commented May 14, 2020

R installed via conda already breaks this paradigm because it is a local installation for only one user.

That's not necessarily true. People can use a conda installation as system if they want. (Maybe in /opt/conda)

When I activate a conda environment that has R installed, it is because I am specifically trying to isolate my analysis from whatever I happened to have installed on that computer

That maybe your use case, but it might just be that a user just wants to easily install R.

If the activate script only warns about the presence of a user directory, then that leaves me to try to come up with some way to get conda R to ignore it (and then I have to help my collaborators do the same).

The script can give specific instructions on how to fix this.

@ebolyen
Copy link

ebolyen commented May 14, 2020

Hi @isuruf,
I see what you are saying here, and I get it, but it really does seem (to me) that the costs of following upstream precisely seriously outweigh the benefits.

People can use a conda installation as system if they want.

I am very skeptical that this is a use-case that is worth optimizing for. I would argue that many R users simply follow RStudios instructions which incidentally does not play very well with non-default installation paths (such as what a base conda install would create without trickery).

In the event the installer of R via conda is able to use trickery, then it seems that creating a simple workaround to "enable" user-local packages would be an appropriate escape hatch, but again I am skeptical of this use case.


In contrast, I have met many people who have been interested in systems which package R libraries as part of a larger conda environment, and these same people often have a system install of R (with RStudio) and some of the same packages. By default, the local-packages are loaded first, which causes very inscrutable binary incompatibilities. This is not only intimidating, but it breaks their environment in a manner which is not obvious.

Perhaps a compromise here would be to load user-local packages last, but again this would deviate from upstream, and you would be unable to override "system" (conda) installed packages as is often the intent of user-local.


Python has a very similar problem here, using essentially the same package management strategy, however they also now have a built-in environment management tool, which is specifically specced to ignore user-local packages (providing a site-flag to load them if desired).

The major difference is that R installs into user-local packages by default, whereas the default for Python has been, for a long time, to install into the site-local packages. So this incompatibility with conda was generally unnoticed. That said, pip had intended (and I have not looked recently) to install into user-local by default to prevent people from using sudo. This is going to be a serious problem for conda, and it is an obvious reason for why the built-in environment manager considers user-local packages to not be a part of the environment.

Of course, R is not Python, but I really do see their package management systems as very similar.


Sorry for the long reply, and thanks @jdblischak for trying to move this forward. I still think it's a great idea, but I think this is the same kind of resistance I saw on the miniconda issue tracker from years ago when it was first discussed.

I've resorted to creating pre/post-activation hooks which just do the following:

export R_LIBS_USER=$CONDA_PREFIX/lib/R/library/
export PYTHONNOUSERSITE=$CONDA_PREFIX/lib/python*/site-packages/

The approach in this PR is much more polite than that.

@isuruf
Copy link
Member

isuruf commented May 14, 2020

I'm glad you brought up python.

Python has a very similar problem here, using essentially the same package management strategy, however they also now have a built-in environment management tool, which is specifically specced to ignore user-local packages (providing a site-flag to load them if desired).

This is not true. With python, libraries in $HOME/.local are loaded even in a conda environment
When I install a library in $HOME/.local I expect it to work in all python environments with the same major.minor version

@ebolyen
Copy link

ebolyen commented May 14, 2020

I was making that same point, I think we just disagree on what is or is not a "problem". Python's environment manager ignores user-local, so in this way conda deviates from upstream (if you consider it an environment manager).

@isuruf
Copy link
Member

isuruf commented May 14, 2020

Python's environment manager ignores user-local, so in this way conda deviates from upstream (if you consider it an environment manager).

I don't understand this. How does conda deviate from upstream?

@ebolyen
Copy link

ebolyen commented May 14, 2020

I don't understand this. How does conda deviate from upstream?

This section of PEP 405 defines the behavior of venv the built-in environment manager w.r.t. user-local packages.

EDIT: there was a better subsection to link to

@ebolyen
Copy link

ebolyen commented May 14, 2020

I acknowledge that as both a package manager and environment manager, it may be impossible for conda to perfectly handle both of these situations as there isn't a strong distinction between a "root" environment and a normal environment. So it is difficult to really disentangle these competing situations.

@dpryan79
Copy link
Contributor

I've needed to use both cases for R. For example, I configured the rstudio server at my old institute to use various versions of conda-installed r-base. This worked quite well and was much more convenient than compiling and installing various versions manually. Users then used the normal methods to install packages in their home directories (the default locations).

At the same time our pipelines used r-base in preconfigured envs and then just had the following in their Rscripts:

.libPaths(R.home("library"))

Then users could run scripts and everything in their home directories would be ignored. So you don't need an environment variable, if you want only-conda libraries loaded just add that one line.

@jdblischak
Copy link
Member Author

That's not necessarily true. People can use a conda installation as system if they want. (Maybe in /opt/conda)

@isuruf They could use sudo to install conda as system, and thus prevent themselves and others from installing packages there. But I would argue that is the less common case. In the conda installation instructions, it states:

We recommend you install Anaconda for the local user, which does not require administrator permissions and is the most robust type of installation. You can also install Anaconda system wide, which does require administrator permissions.

Thus I'd prefer the default behavior to work well with the recommended installation guidelines, and then provide workarounds for the non-standard system installation of conda (especially since a systems administrator installing R for a group of users should be comfortable with the extra steps).

The script can give specific instructions on how to fix this.

@isuruf Do you have suggestions for the workaround?

Here is a potential workaround I came up with. Users could put the following in their .Rprofile. If R is installed with conda, this will configure the library paths to only point to the conda R package directory:

local({
  conda <- Sys.getenv("CONDA_PREFIX", unset = NA)
  if (!is.na(conda) && startsWith(R.home(), conda)) {
    .libPaths(R.home("library"))
  }
})

Then users could run scripts and everything in their home directories would be ignored. So you don't need an environment variable, if you want only-conda libraries loaded just add that one line.

@dpryan79 While this works, I don't like that this solution requires R to be installed with conda. I don't think an R script should be concerned with implementation details like whether or not R was installed via conda or not.

@isuruf
Copy link
Member

isuruf commented May 14, 2020

Here is a potential workaround I came up with.

That sounds like a good idea and maybe a conda-forge package just for that might fix this. We can warn in the activate script here and ask the user to install that package.

@jamesmyatt
Copy link

The following workaround doesn't work on Windows, because R.home uses '/' separators and Sys.getenv uses '\\' separators. Also I think you only need .libPaths("").

local({
  conda <- Sys.getenv("CONDA_PREFIX", unset = NA)
  if (!is.na(conda) && startsWith(R.home(), conda)) {
    .libPaths(R.home("library"))
  }
})

@jdblischak
Copy link
Member Author

The following workaround doesn't work on Windows, because R.home uses '/' separators and Sys.getenv uses '\' separators.

@jamesmyatt Thanks for reporting this shortcoming with the proposed solution. Below is an updated version that should work on both Windows and Unix-alike operating systems. I tested it on Windows 10.

local({
  conda <- Sys.getenv("CONDA_PREFIX", unset = NA)
  rhome <- normalizePath(R.home())
  if (!is.na(conda) && startsWith(rhome, conda)) {
    .libPaths(R.home("library"))
  }
})

Also I think you only need .libPaths("").

You're correct that .libPaths("") and .libPaths(R.home("library")) produce the same result, so you can use whichever you like. I prefer the latter since it is more explicit about the change being made. The former requires knowing that .libPaths() returns unique(c(new, .Library.site, .Library)).

@jdblischak
Copy link
Member Author

A quick update on this PR: As seen above, there is a debate about whether this "isolation" feature should be implemented. The main issue is that because conda is used in so many different ways, it is difficult to find the right behavior that will be intuitive for all of the use cases.

I've abandoned this effort for now because the situation is actually fixed well enough for me on Linux. I don't know why or how the change was made, but now the conda recipe changes the value of the "platform" variable when installing R on Linux.

# System R installed from CRAN via apt-get on Ubuntu
> R.version$platform
[1] "x86_64-pc-linux-gnu"

# R installed from conda-forge
> R.version$platform
[1] "x86_64-conda_cos6-linux-gnu"

Because the default path to the user library uses this platform variable, when I activate a conda environment in Ubuntu, it ignores my user packages. Thus it's properly isolated.

Unfortunately conda R is still not isolated in the following situations:

  1. The user manually sets a custom path to their R user library via R_LIBS_USER.

  2. On Windows. This is because the platform variable is the same. It would be nice to insert "conda" into the platform like in the Linux case, but I don't know how to do this. This is already how it works on Linux, so it would be more consistent if Windows did the same.

    # System R installed from CRAN on Windows
    > R.version$platform
    [1] "x86_64-w64-mingw32"
    
    # R installed from conda-forge
    > R.version$platform
    [1] "x86_64-w64-mingw32"
    

My macbook died, so I don't know what the current behavior is on macOS. I'd appreciate it if someone could run R.version$platform with the system and conda versions of R to see if these are the same or different on macOS.

@mfansler
Copy link
Member

Might be worth mentioning that the conda-ecosystem-user-package-isolation package provides activation scripts for isolating Python and R from user-level packages. For users that want this as default behavior, they could add it to their create_default_packages.

@jdblischak
Copy link
Member Author

Might be worth mentioning that the conda-ecosystem-user-package-isolation package provides activation scripts for isolating Python and R from user-level packages. For users that want this as default behavior, they could add it to their create_default_packages.

@mfansler Yes! That is very relevant. It's a shame that it's been around for over 3 years, and I had no idea about it. Unfortunately I assume the only way to really learn about it is through word of mouth. Thanks for sharing!

To clarify though, this solves the problem in the situation where a user has explicitly set R_LIBS_USER. However, it won't help in the existing situation on Windows (and likely macOS) where the default user library is defined by R itself (which is addressed by this PR). If a user has any packages installed in the default directory, conda-installed R will still search there first. The only reason this is no longer a problem on Linux is, as I mentioned above, it somehow inserts "conda" into the platform name.

Lastly, for the benefits of others that like me were not aware of create_default_packages, it is documented here.

@jdblischak
Copy link
Member Author

My macbook died, so I don't know what the current behavior is on macOS. I'd appreciate it if someone could run R.version$platform with the system and conda versions of R to see if these are the same or different on macOS.

I currently have access to a macOS (Catalina 10.15.7) machine through work. I checked the platform, and unfortunately "conda" is not inserted into the name. However, you could get lucky if your version of macOS differs from the version in which the r-base binary was built. In my case, the user-installed packages wouldn't conflict, but they very easily could on someone else's machine.

# homebrew-installed R
# R 4.1.3
> R.version$platform
[1] "x86_64-apple-darwin19.6.0"

# conda-installed R
# r-base==4.1.2=h2b051ba_0
> R.version$platform
[1] "x86_64-apple-darwin13.4.0"

@jdblischak
Copy link
Member Author

A few updates relevant to this PR:

  • In the upcoming release of R 4.2.0, the default user library on Windows is being changed to the Local Application Data directory. My test scripts assume the user directory is in Documents/

The default personal library on Windows, folder ‘R\win-library\x.y’ where ‘x.y’ stands for R release ‘x.y.z’, is now a subdirectory of Local Application Data directory (usually a hidden directory ‘C:\Users\username\AppData\Local’). Use shell.exec(.libPaths()[1]) from R to open the personal library in Explorer when it is first in the list (PR#17842).

https://github.com/wch/r-source/blob/tags/R-4-2-0/doc/NEWS.Rd#L324
wch/r-source@9196047

  • The /etc/Renviron.in file has changed since my original PR against R 4.0.0. My patch won't even work any more

wch/r-source@tags/R-4-0-0...tags/R-4-2-0#diff-1cbaac4768fd110525ba9086cb7a684aaf2c6555389c5446c913effbfec90c85

I'm not sure what to do with this PR. On the one hand, I'm aware at this point that it is going on almost 4 years, and has zero chance of being merged. Thus I'm not planning on updating it. On the other hand, I think it serves as very useful documentation. Users of conda-installed R are typically surprised when they find out that their conda envrionments are using their locally installed packages (and are thus not at all isolated or reproducible). I know I was. Furthermore, this PR makes it clear that it's not enough to simply temporarily clear R_LIBS_USER (this is obviously a great start, and is now possible with conda-ecosystem-user-package-isolation-feedstock). You also have to prevent R from looking for packages in the default user location. Otherwise, any time a user installs the same R version with conda as they already have installed on their machine, conda-installed R will use the local user packages instead of the conda binaries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet