Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pass P3M repos to pak, if they should be used #1678

Closed
gaborcsardi opened this issue Sep 6, 2023 · 10 comments
Closed

Pass P3M repos to pak, if they should be used #1678

gaborcsardi opened this issue Sep 6, 2023 · 10 comments
Labels
bug an unexpected problem or unintended behavior install 🧺

Comments

@gaborcsardi
Copy link

Right now, even if I have

> getOption("repos")
                                                         CRAN
"https://packagemanager.posit.co/cran/__linux__/jammy/latest"

renv passed this to pak:

$repos
                                         CRAN
"https://packagemanager.posit.co/cran/latest"

so pak will compile packages from source, instead of installing the binaries.
This was referenced Sep 6, 2023
Open
Closed
@kevinushey
Copy link
Collaborator

That transformation should be happening here:

renv/R/pak.R

Lines 67 to 71 in c60722d

# transform repositories
if (renv_ppm_enabled()) {
repos <- getOption("repos")
renv_scope_options(repos = renv_ppm_transform(repos))
}

Can you share any more details / a reproducible example?

@gaborcsardi
Copy link
Author

gaborcsardi commented Sep 6, 2023 •
edited
Loading

Well, I can't share a fully reproducible example (i.e. Dockerfile), because init() fails in non-interactive mode (#1676).

But here is one with an already existing project:

FROM rocker/r-ver:4.3.1

WORKDIR /root

RUN apt-get update && apt-get install -y git

RUN git clone -b renv2 https://github.com/r-lib/sessioninfo

WORKDIR /root/sessioninfo

RUN echo 'RENV_CONFIG_PAK_ENABLED=true' > .Renviron

RUN R -q -e 'renv:::renv_pak_init(stream = "devel", force = TRUE)'

RUN R -q -e 'print(getOption("repos")); trace(pak:::remote, quote(print(getOption("repos")))); renv::restore()'

repos is set to:

#11 [8/8] RUN R -q -e 'print(getOption("repos")); trace(pak:::remote, quote(print(getOption("repos")))); renv::restore()'
#11 1.341 - Project '~/sessioninfo' loaded. [renv 1.0.2]
#11 1.349 - None of the packages recorded in the lockfile are currently installed.
#11 1.388 > print(getOption("repos")); trace(pak:::remote, quote(print(getOption("repos")))); renv::restore()
#11 1.390                                                          CRAN
#11 1.390 "https://packagemanager.posit.co/cran/__linux__/jammy/latest"
#11 1.390                                                           P3M
#11 1.390 "https://packagemanager.posit.co/cran/__linux__/jammy/latest"

But then when pak calls the subprocess, repos is (temporarily) set to

#11 1.407 Tracing function "remote" in package "pak (not-exported)"
#11 1.408 [1] "remote"
#11 1.860 Tracing remote(function(...) get("pkg_install_make_plan", asNamespace("pak"))(...),  .... on entry
#11 1.860                                          CRAN
#11 1.860 "https://packagemanager.posit.co/cran/latest"
#11 1.860                                           P3M
#11 1.860 "https://packagemanager.posit.co/cran/latest"

And you can also see that cli is installed from source:

#11 9.101 ℹ Getting 1 pkg with unknown size
#11 9.951 ✔ Got cli 3.6.1 (source) (565.37 kB)
#11 10.03 ℹ Building cli 3.6.1
#11 29.45 ✔ Built cli 3.6.1 (18.2s)
#11 30.58 ✔ Installed cli 3.6.1  (1.1s)
#11 30.59 ✔ 1 pkg: added 1, dld 1 (565.37 kB) [28.7s]

Maybe because that's also the URL in the renv.lock file: https://github.com/r-lib/sessioninfo/blob/renv2/renv.lock#L7

But I am fairly sure that when the lock file was created repos was also set to the Linux specific URLs, and renv rewrote that to the generic P3M URL.

@kevinushey kevinushey added bug an unexpected problem or unintended behavior install 🧺 labels Sep 6, 2023
@kevinushey kevinushey added this to the 1.0.3 milestone Sep 6, 2023
@kevinushey
Copy link
Collaborator

Is this on x86_64 or aarch64?

@gaborcsardi
Copy link
Author

x86_64. If I add

RUN R -q -e 'pak::pkg_install("cli?reinstall")'

at the end, it installs the binary:

#12 [9/9] RUN R -q -e 'pak::pkg_install("cli?reinstall")'
#12 1.508 - Project '~/sessioninfo' loaded. [renv 1.0.2]
#12 1.675 > pak::pkg_install("cli?reinstall")
#12 3.063
#12 4.490 ✔ Updated metadata database: 2.67 MB in 3 files.
#12 4.491
#12 4.493 ℹ Updating metadata database
#12 7.875 ✔ Updating metadata database ... done
#12 7.876
#12 8.089
#12 8.094 → Will install 1 package.
#12 8.133 → The package (0 B) is cached.
#12 8.137 + cli   3.6.1
#12 8.138
#12 8.289 ℹ No downloads are needed, 1 pkg is cached
#12 9.169 ✔ Got cli 3.6.1 (x86_64-pc-linux-gnu-ubuntu-22.04) (1.27 MB)
#12 10.45 ✔ Installed cli 3.6.1  (1.1s)
#12 10.46 ✔ 1 pkg: added 1, dld 1 (1.27 MB) [8.8s]
#12 10.47 >
#12 10.47 >
#12 DONE 10.7s

@kevinushey
Copy link
Collaborator

Thanks! I think I understand what's going on now. I'll take a look.

@kevinushey
Copy link
Collaborator

Should be fixed via b5162ca.

@gaborcsardi
Copy link
Author

gaborcsardi commented Sep 7, 2023 •
edited
Loading 

FROM rocker/r-ver:4.3.1

WORKDIR /root

RUN apt-get update && apt-get install -y git

RUN git clone -b renv3 https://github.com/r-lib/sessioninfo

WORKDIR /root/sessioninfo

RUN echo 'RENV_CONFIG_PAK_ENABLED=true' > .Renviron

RUN R -q -e 'renv:::renv_pak_init(stream = "devel", force = TRUE)'

RUN R -q -e 'print(getOption("repos")); trace(pak:::remote, quote(print(getOption("repos")))); renv::restore()'

RUN R -q -e 'pak::pkg_install("cli?reinstall")'
docker build --platform linux/amd64 --progress plain .

[...]
#11 9.303 → Will install 1 package.
#11 9.334 → Will download 1 package with unknown size.
#11 9.338 + cli   3.6.1 [cmp][dl]
#11 9.340
#11 9.454 Tracing remote(function(...) get("pkg_install_do_plan", asNamespace("pak"))(...),  .... on entry
#11 9.454                                                          CRAN
#11 9.454 "https://packagemanager.posit.co/cran/__linux__/jammy/latest"
#11 9.454                                                           P3M
#11 9.454 "https://packagemanager.posit.co/cran/__linux__/jammy/latest"
#11 9.546 ℹ Getting 1 pkg with unknown size
#11 10.37 ✔ Got cli 3.6.1 (x86_64-pc-linux-gnu) (1.27 MB)
#11 11.51 ✔ Installed cli 3.6.1  (1.1s)
#11 11.52 ✔ 1 pkg: added 1, dld 1 (1.27 MB) [9.5s]
[...]

I am not sure why pak thinks that it would download a source package, that is not ideal, but at least it does download a binary one, so this is a big improvement.

@kevinushey
Copy link
Collaborator

I am not sure why pak thinks that it would download a source package, that is not ideal, but at least it does download a binary one, so this is a big improvement.

My guess is that this is because PPM basically works by disguising binary packages as source packages on Linux, so everything in R "thinks" it's a source package, but R CMD INSTALL is smart enough to see that the downloaded package is a binary and so does the right thing.

@gaborcsardi
Copy link
Author

Sure, but pak detects P3M from the repo's headers and then downloads the metadata for the binary packages. E.g. if I call it directly, then it knows that it is installing a binary package for Ubuntu 22.04, as in the last step of the Dockerfile above:

#12 [9/9] RUN R -q -e 'pak::pkg_install("cli?reinstall") '
#12 1.610 - Project '~/sessioninfo' loaded. [renv 1.0.2.9000; sha: 9eb8777]
#12 1.809 > pak::pkg_install("cli?reinstall")
#12 3.315 ℹ Loading metadata database
#12 4.201 ✔ Loading metadata database ... done
#12 4.202
#12 4.437
#12 4.445 → Will install 1 package.
#12 4.498 → Will download 1 package with unknown size.
#12 4.503 + cli   3.6.1 [dl]
#12 4.505
#12 4.666 ℹ Getting 1 pkg with unknown size
#12 5.968 ✔ Got cli 3.6.1 (x86_64-pc-linux-gnu-ubuntu-22.04) (1.27 MB)
#12 7.246 ✔ Installed cli 3.6.1  (1.2s)
#12 7.256 ✔ 1 pkg: added 1, dld 1 (1.27 MB) [5.4s]
#12 7.258 >
#12 7.258 >
#12 DONE 7.4s

@kevinushey
Copy link
Collaborator

Oh, interesting! I do not know the answer to that then :-/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug an unexpected problem or unintended behavior install 🧺
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gaborcsardi @kevinushey