Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get R from RStudio provided apt packages (.deb files) #1161

Merged
merged 21 commits into from
Jul 1, 2022

Conversation

yuvipanda
Copy link
Collaborator

@yuvipanda yuvipanda commented Jun 3, 2022

  • Instructions at https://docs.rstudio.com/resources/install-r/
  • Simpler than getting it from CRAN apt repositories, as it's a single deb that is
    downloaded on a per-version basis rather than an entire apt repository with
    all the quirks that entails.
  • RStudio provides debs all R versions for Ubuntu 20.04, 18.04 and 22.04 -
    making it much easier for us to switch to a different base Ubuntu image (see
    Let FROM <base_image> in the Dockerfile template be configurable #909)
  • The packages are built from https://github.com/rstudio/r-builds, so even if RStudio
    stops providing them in the future, they can be built.
  • Better compatibility with binary builds from packagemanager.rstudio.com (compared to
    R from conda), as these are appropriate deb builds for particular Ubuntu versions that
    also come from rstudio.
  • We just pass through full versions to the RStudio CDN - so new R versions will now
    be available as soon as RStudio hosts them! No more changes required in repo2docker
    when a new R version is available. This means this PR also adds support for R 4.2.
    This is awesome!
  • We no longer hardcode any ubuntu or distro identifiers in the code!

@yuvipanda yuvipanda marked this pull request as ready for review June 3, 2022 19:32
@yuvipanda yuvipanda requested review from minrk, manics, consideRatio and betatim and removed request for minrk June 3, 2022 19:42
@yuvipanda
Copy link
Collaborator Author

This seems to break RStudio < 4.0 :(

@yuvipanda
Copy link
Collaborator Author

Nah, RStudio < 4.0 was already broken lol. Fixed by #1148.

Had to tell rstudio exactly where R is by setting R_HOME: 0fed94c

Also, it rstudio will fail to start if the uid is < 1000. And on my mac, my uid was 502! So I had to test repo2docker with a --user-id=1000 flag to get it to work. Took me a long time.

Might be simpler than getting it from R project managed
apt packages
Other wise our semver package cries
It fetches dependencies and installs them too, unlike dpkg
apt also seems to fetch dependencies when needed
We want users to be able to install new versions of R
as soon as they are out, without us having to do anything
special in repo2docker. So we can't actually check for 'invalid'
R versions. This is ok, as we don't have to special case apt repos
or similar when installing R from rstudio's debs
2020 had an older version of testthat that ran into issues

> # Error in get(genname, envir = envir) : object 'testthat_print' not found
This allows us to dynamically change the base image
without having to manually change the version string here
As Dockerfile executes with /bin/sh, not /bin/bash
@yuvipanda
Copy link
Collaborator Author

ok, this is ready for review! It's the last thing left to do before we can make the base image configurable.

Copy link
Member

@manics manics left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! I've done a code review, but I haven't done any functional testing.

repo2docker/buildpacks/r.py Outdated Show resolved Hide resolved
repo2docker/buildpacks/r.py Outdated Show resolved Hide resolved
# it is - just $PATH isn't enough. I discovered these are the env vars it
# looks for by digging through RStudio source and finding https://github.com/rstudio/rstudio/blob/main/src/cpp/r/session/RDiscovery.cpp
("R_HOME", f"/opt/R/{self.r_version}/lib/R"),
("R_DOC_DIR", "${R_HOME}/doc"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is ${R_HOME} expanded to f"/opt/R/{self.r_version}/lib/R" at runtime? Or is it actually completely separate?
If it's the former I think it'd be clearer to use the expanded variable here, if it's the latter would you mind adding a comment explaining the difference?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manics it gets expanded at runtime. I think it's actually clearer to let it expand at runtime, as you can see that these are dependent if you run env while you are using the image.

tests/r/simple/runtime.txt Outdated Show resolved Hide resolved
@@ -7,20 +7,6 @@
from repo2docker import buildpacks


def test_unsupported_version(tmpdir):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the removal of this test mean all versions are supported?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YESSS! basically anything that has a deb produced by RStudio is now supported

yuvipanda and others added 2 commits June 22, 2022 13:40
Co-authored-by: Simon Li <orpheus+devel@gmail.com>
Let's remove unnecessary changes
Co-authored-by: Simon Li <orpheus+devel@gmail.com>
@yuvipanda
Copy link
Collaborator Author

Thanks a lot for the review, @manics! I think I've addressed your comments now

Copy link
Member

@consideRatio consideRatio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments to be addressed, but otherwise looks amazing!!

@yuvipanda
Copy link
Collaborator Author

@consideRatio ty, I've accounted for both your suggestions i think.

Copy link
Member

@consideRatio consideRatio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An update of the title to reflect using downloaded deb files and this is ready for merge i think. (I've not tested this, just reviewed code.)

@manics
Copy link
Member

manics commented Jun 23, 2022

How risky do you think this change is to users? Are you happy for it to go into a tag/release in the near future, or do you think it needs extended testing on mybinder.org first?

@yuvipanda yuvipanda changed the title Get R from RStudio managed apt packages Get R from RStudio provided apt packages (.deb files) Jun 23, 2022
@yuvipanda
Copy link
Collaborator Author

@consideRatio done!

@manics in general i always like testing in repo2docker for a week or two before a release :D

@yuvipanda
Copy link
Collaborator Author

@consideRatio @manics anything else i can do to help get this merged? :D

@consideRatio consideRatio merged commit ae8ed29 into jupyterhub:main Jul 1, 2022
@consideRatio
Copy link
Member

I saw no unresolved comments and this has been open for a long time so i figure a merge is in order! Thanks Yuvi and good that you pinged!!

@yuvipanda
Copy link
Collaborator Author

Yay thanks a lot for the merge, @consideRatio!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants