Upgrade user image #62
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yuvipanda
added a commit
to yuvipanda/datahub-old-fork
that referenced
this pull request
Apr 17, 2021
Earlier, interactive use of R somehow wasn't loading appropriate instructions on setting CRAN from Rprofile.site, resulting (partially) in errors like this: berkeley-dsep-infra/datahub#2255 (comment) My informed guess is that the `local{}` was scoping the effects, so it worked during build but not during interactive use. Removing that seems to make it work for both. Inspired by toolforge/paws#62
|
I dove into a rabbithole here about R default mirrors, and it helped me fix a long standing issue in my current job (berkeley-dsep-infra/datahub#2316), so yay! |
|
Thanks for this @yuvipanda. |
| ENV PATH=/srv/paws/pwb:/srv/paws/bin:/srv/paws:$PATH | ||
|
|
||
| USER ${NB_USER} | ||
| RUN python3.6 -m venv /srv/paws | ||
| RUN python3 -m venv /srv/paws | ||
|
|
||
| # Install base notebook packages | ||
| RUN pip install --no-cache-dir \ | ||
| jupyterhub==1.1.0 \ |
There was a problem hiding this comment.
any reason to keep 1.1.0 here?
There was a problem hiding this comment.
Since the chart itself is running z2jh 0.9, which is jupyterhub 1.1. Once the chart is upgraded, this can be too
chicocvenancio
previously approved these changes
Apr 19, 2021
- Bump to Ubuntu 20.04. This brings in lots of newer packages, but particularly Python 3.8 (https://phabricator.wikimedia.org/T265957). Many new packages are 3.7+ only. - Use upstream R deb packages (https://cran.r-project.org/bin/linux/ubuntu/), than those from Ubuntu. This is maintained by the same folks who do R in debian. This brings us R 4.x (https://phabricator.wikimedia.org/T268923), but more importantly enables binary package installation. - Use packagemanager.rstudio.com to install R packages. This is a huge benefit - it can install any version of packges, and it has binary package builds for most of them! These binary packages are built against focal + the upstream R debs. See announcement info here: https://blog.rstudio.com/2020/07/01/announcing-public-package-manager/. This reduces the user image build time a *lot* - Put R packages in /srv/r, to match /srv/paws for our venv. Users can more easily install R packages temporarily now, and we don't mess with things under `/usr`. It also simplifies our Dockerfile, reducing number of times we have to switch to root. - Redirect most stdout from apt-get to /dev/null. We still get stderr, so no real functionality lost. - Remove some explicit nbextension enable / install commands. These are no longer necessary. - Specify a loose pin for notebook & jupyterlab versions.
chicocvenancio
approved these changes
May 22, 2021
|
I can merge and rebuild the image later if you like (will be fixing the deployhook eventually soon--promise!) |
|
I'll try to merge and deploy. |
Merged
shaneknapp
pushed a commit
to berkeley-dsep-infra/biology-user-image
that referenced
this pull request
Sep 6, 2024
Earlier, interactive use of R somehow wasn't loading appropriate instructions on setting CRAN from Rprofile.site, resulting (partially) in errors like this: berkeley-dsep-infra/datahub#2255 (comment) My informed guess is that the `local{}` was scoping the effects, so it worked during build but not during interactive use. Removing that seems to make it work for both. Inspired by toolforge/paws#62
shaneknapp
pushed a commit
to berkeley-dsep-infra/datahub-user-image
that referenced
this pull request
Sep 25, 2024
Earlier, interactive use of R somehow wasn't loading appropriate instructions on setting CRAN from Rprofile.site, resulting (partially) in errors like this: berkeley-dsep-infra/datahub#2255 (comment) My informed guess is that the `local{}` was scoping the effects, so it worked during build but not during interactive use. Removing that seems to make it work for both. Inspired by toolforge/paws#62
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
but particularly Python 3.8 (https://phabricator.wikimedia.org/T265957).
Many new packages are 3.7+ only.
than those from Ubuntu. This is maintained by the same folks who
do R in debian. This brings us R 4.x (https://phabricator.wikimedia.org/T268923),
but more importantly enables binary package installation.
huge benefit - it can install any version of packges, and it has
binary package builds for most of them! These binary packages are
built against focal + the upstream R debs. See announcement info
here: https://blog.rstudio.com/2020/07/01/announcing-public-package-manager/.
This reduces the user image build time a lot
can more easily install R packages temporarily now, and we don't
mess with things under
/usr. It also simplifies our Dockerfile,reducing number of times we have to switch to root.
so no real functionality lost.
no longer necessary.