feat(installation): download lib binaries #191

eitsupi · 2023-09-28T15:29:06Z

No description provided.

This reverts commit 2a3c2e2.

eitsupi · 2023-09-29T11:59:59Z

It seems working! (I tested on Raspberry Pi4)

$ docker run --rm -it rocker/r-ver bash
root@0e033cfdace2:/# R

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages("remotes")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://packagemanager.posit.co/cran/latest/src/contrib/remotes_2.4.2.1.tar.gz'
Content type 'binary/octet-stream' length 152858 bytes (149 KB)
==================================================
downloaded 149 KB

* installing *source* package ‘remotes’ ...
** package ‘remotes’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (remotes)

The downloaded source packages are in
        ‘/tmp/RtmpE8wffs/downloaded_packages’
> remotes::install_github("eitsupi/prqlr#191")
Downloading GitHub repo eitsupi/prqlr@lib-bin-install
Running `R CMD build`...
* checking for file ‘/tmp/RtmpE8wffs/remotesa570410f6/eitsupi-prqlr-4f2c220/DESCRIPTION’ ... OK
* preparing ‘prqlr’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘prqlr_0.5.3.9000.tar.gz’
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘prqlr’ ...
** using staged installation
----------------------- [RUST NOT FOUND]---------------------------
The 'cargo' command was not found on the PATH. Please install rustc
from: https://www.rust-lang.org/tools/install

Alternatively, you may install cargo from your OS package manager:
 - Debian/Ubuntu: apt-get install cargo
 - Fedora/CentOS: dnf install cargo
 - macOS: brew install rustc
-------------------------------------------------------------------

ERROR: configuration failed for package ‘prqlr’
* removing ‘/usr/local/lib/R/site-library/prqlr’
Warning message:
In i.p(...) :
  installation of package ‘/tmp/RtmpE8wffs/filea6ae0f25/prqlr_0.5.3.9000.tar.gz’ had non-zero exit status
> Sys.setenv(NOT_CRAN="true")
> remotes::install_github("eitsupi/prqlr#191")
Downloading GitHub repo eitsupi/prqlr@lib-bin-install
Running `R CMD build`...
* checking for file ‘/tmp/RtmpE8wffs/remotesa6d1aaa1f/eitsupi-prqlr-4f2c220/DESCRIPTION’ ... OK
* preparing ‘prqlr’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘prqlr_0.5.3.9000.tar.gz’
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘prqlr’ ...
** using staged installation
Try to download pre-built binary...
Found pre built binary at <https://github.com/eitsupi/prqlr/releases/download/lib-0.1.0/libprqlr-0.1.0-aarch64-unknown-linux-musl.tar.gz>.
Downloading...
Checking SHA256 hash to <a81a8643eb6dce90c249dbd704759759d54a295b38cc23525e8b1aa1da9bd570> for </tmp/RtmpF83Xkn/file25f2b9e26ab.tar.gz>...
SHA256 matches for </tmp/RtmpF83Xkn/file25f2b9e26ab.tar.gz>.
Extracted pre built binary to <tools> directory.
----------------------- [LIBRARY FOUND]---------------------------
The library was found in the tools directory. No need to build it.
-------------------------------------------------------------------

** libs
using C compiler: ‘gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
rm -Rf "prqlr.so" "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/target//release/libprqlr.a" "entrypoint.o"
gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include    -fPIC  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c entrypoint.c -o entrypoint.o
if [ -f "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/../tools/libprqlr.a" ]; then \
        mkdir -p "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/target//release" ; \
        mv "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/../tools/libprqlr.a" "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/target//release/libprqlr.a" ; \
        exit 0; \
fi && \
if [ -f "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/vendor.tar.xz" ]; then \
        mkdir -p "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/vendor" && \
        /usr/bin/tar --extract --xz --file "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/vendor.tar.xz" -C "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/vendor" && \
        mkdir -p "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/.cargo" && \
        cp "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/vendor-config.toml" "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/.cargo/config.toml"; \
fi && \
if [ "true" != "true" ]; then \
        export CARGO_HOME="/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/.cargo"; \
        export CARGO_BUILD_JOBS=2; \
fi && \
        export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.cargo/bin" && \
        cargo build --lib --manifest-path="/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/Cargo.toml" --target-dir "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/target" --target="" \
                --profile="release" --features=""
if [ "true" != "true" ]; then \
        rm -Rf "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/.cargo" "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/vendor" "/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/target//release/build"; \
fi
gcc -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o prqlr.so entrypoint.o -L/tmp/Rtmp6r0UVv/R.INSTALL1a47cc2ea4c/prqlr/src/rust/target//release -lprqlr -L/usr/local/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-prqlr/00new/prqlr/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (prqlr)
> prqlr::prql_version()
[1] ‘0.9.5’

tools/prep-lib.R

eitsupi · 2023-09-30T01:07:44Z

It works!

$ docker run --rm -it rocker/r-ver:4.2 bash
Unable to find image 'rocker/r-ver:4.2' locally
4.2: Pulling from rocker/r-ver
b2837baf7808: Pull complete
68e6ffd68fb4: Pull complete
4a891d979184: Pull complete
74c293d49c09: Pull complete
a582ad5cfd1b: Pull complete
Digest: sha256:7c89f3b30b3f998215a74bfd992af90346f34b4f3d233cdfb0c93634f3b9ecf4
Status: Downloaded newer image for rocker/r-ver:4.2
root@bc19b35df54a:/# R

R version 4.2.3 (2023-03-15) -- "Shortstop Beagle"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages("remotes"); Sys.setenv(NOT_CRAN="true"); remotes::install_github("eitsupi/prqlr#191")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://packagemanager.posit.co/cran/2023-04-20/src/contrib/remotes_2.4.2.tar.gz'
Content type 'binary/octet-stream' length 152848 bytes (149 KB)
==================================================
downloaded 149 KB

* installing *source* package ‘remotes’ ...
** package ‘remotes’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (remotes)

The downloaded source packages are in
        ‘/tmp/RtmpzdkGjv/downloaded_packages’
Downloading GitHub repo eitsupi/prqlr@lib-bin-install
Running `R CMD build`...
* checking for file ‘/tmp/RtmpzdkGjv/remotes85d8b4840/eitsupi-prqlr-f1b6356/DESCRIPTION’ ... OK
* preparing ‘prqlr’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘prqlr_0.5.3.9000.tar.gz’
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘prqlr’ ...
** using staged installation
Try to download pre-built binary...
Found pre built binary at <https://github.com/eitsupi/prqlr/releases/download/lib-0.1.0/libprqlr-0.1.0-aarch64-unknown-linux-musl.tar.gz>.
Downloading...
Checking SHA256 for </tmp/RtmpsKQ9P8/filea1275e7775.tar.gz>...
SHA256 matches for </tmp/RtmpsKQ9P8/filea1275e7775.tar.gz>.
Extracted pre built binary to <tools> directory.

----------------------- [LIBRARY FOUND]----------------------------
The library was found at tools/libprqlr.a. No need to build it.
-------------------------------------------------------------------

** libs
rm -Rf "prqlr.so" "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/target//release/libprqlr.a" "entrypoint.o"
gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c entrypoint.c -o entrypoint.o
if [ -f "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/../tools/libprqlr.a" ]; then \
        mkdir -p "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/target//release" ; \
        mv "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/../tools/libprqlr.a" "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/target//release/libprqlr.a" ; \
        exit 0; \
fi && \
if [ -f "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/vendor.tar.xz" ]; then \
        mkdir -p "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/vendor" && \
        /usr/bin/tar --extract --xz --file "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/vendor.tar.xz" -C "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/vendor" && \
        mkdir -p "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/.cargo" && \
        cp "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/vendor-config.toml" "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/.cargo/config.toml"; \
fi && \
if [ "true" != "true" ]; then \
        export CARGO_HOME="/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/.cargo"; \
        export CARGO_BUILD_JOBS=2; \
fi && \
        export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.cargo/bin" && \
        cargo build --lib --manifest-path="/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/Cargo.toml" --target-dir "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/target" --target="" \
                --profile="release" --features=""
if [ "true" != "true" ]; then \
        rm -Rf "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/.cargo" "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/vendor" "/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/target//release/build"; \
fi
gcc -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o prqlr.so entrypoint.o -L/tmp/RtmpoYMl3K/R.INSTALL9640ff82ef/prqlr/src/rust/target//release -lprqlr -L/usr/local/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-prqlr/00new/prqlr/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (prqlr)
> prqlr::prql_version()
[1] ‘0.9.5’

A following up for #191

WIP feat(installation): download lib binaries

d49dc2c

eitsupi added enhancement New feature or request develop labels Sep 28, 2023

eitsupi added 3 commits September 28, 2023 15:37

fix: make executable

d5c94e3

chore: revert not to use Makevars.in

2a3c2e2

Revert "chore: revert not to use Makevars.in"

5613266

This reverts commit 2a3c2e2.

eitsupi mentioned this pull request Sep 28, 2023

Provide R package binaries? beniaminogreen/zoomerjoin#85

Closed

eitsupi added 3 commits September 29, 2023 10:25

chore: refactor

156f1ee

feat: download pre built binary

2d2382c

feat: checksum

4f2c220

eitsupi commented Sep 29, 2023

View reviewed changes

tools/prep-lib.R Outdated Show resolved Hide resolved

tools/prep-lib.R Outdated Show resolved Hide resolved

eitsupi added 4 commits September 30, 2023 00:42

chore: refactoring

3c99bf0

fix: support R 4.2 syntax

25bcb1c

chore: tweak messages

b6cac30

chore: tweak messages and update for windows

f1b6356

fix: update tag name pattern

01bc636

eitsupi force-pushed the lib-bin-install branch 2 times, most recently from c1acabf to 432db0c Compare September 30, 2023 13:46

ci: add tests for installing with bin lib

768d181

eitsupi force-pushed the lib-bin-install branch from 432db0c to 768d181 Compare September 30, 2023 13:49

eitsupi added 7 commits September 30, 2023 13:59

fix: fix to detect amd64

ce37290

build: add the generate-lib-sums task to the Taskfile

3811fce

Merge branch 'main' into lib-bin-install

5dbf5b8

ci: update the Makevars path to Makevars.in

31f3428

Merge branch 'main' into lib-bin-install

db14377

chore: update lib-sums

ab55b30

ci: fix test step

c9dc201

eitsupi changed the title ~~WIP feat(installation): download lib binaries~~ feat(installation): download lib binaries Sep 30, 2023

eitsupi marked this pull request as ready for review September 30, 2023 15:53

eitsupi force-pushed the lib-bin-install branch from cb883c3 to 6362000 Compare September 30, 2023 15:58

chore: tweak messages

03a9fec

eitsupi force-pushed the lib-bin-install branch from 6362000 to 03a9fec Compare September 30, 2023 16:07

eitsupi merged commit 97fa7ce into main Sep 30, 2023
18 checks passed

eitsupi deleted the lib-bin-install branch September 30, 2023 16:24

eitsupi mentioned this pull request Sep 30, 2023

docs: add notes about pre-built libraries #195

Merged

eitsupi added a commit that referenced this pull request Oct 1, 2023

docs: add notes about pre-built libraries (#195)

bea389e

A following up for #191

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(installation): download lib binaries #191

feat(installation): download lib binaries #191

eitsupi commented Sep 28, 2023

eitsupi commented Sep 29, 2023

eitsupi commented Sep 30, 2023

feat(installation): download lib binaries #191

feat(installation): download lib binaries #191

Conversation

eitsupi commented Sep 28, 2023

eitsupi commented Sep 29, 2023

eitsupi commented Sep 30, 2023