unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so': #4532

vnijs · 2022-08-29T23:37:42Z

What happens?

When installing from source on Ubuntu 22.04 (aarch64) using > install.packages('duckdb', repos=c('https://duckdb.r-universe.dev', 'https://cloud.r-project.org')) I see the below:

/usr/bin/ld: /tmp/RtmpCkedt0/R.INSTALL78a33bcebf27/duckdb/src/../inst/include/cpp11/protect.hpp:42: warning: too many GOT entries for -fpic, please recompile with -fPIC

This, I believe is the same issue as: #3049

When installing binary packages from CRAN using the below in a Docker file the install seems to succeed.

RUN R -e "options(HTTPUserAgent = sprintf('R/%s R (%s)', getRversion(), paste(getRversion(), R.version['platform'], R.version['arch'], R.version['os']))); install.packages(c('arrow', 'duckdb'), repos = 'https://packagemanager.rstudio.com/all/__linux__/jammy/latest', Ncpus=-1)"

However, when trying to load the library I get the following error message.

> library(duckdb)
Loading required package: DBI
Error: package or namespace load failed for 'duckdb' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so':
  /usr/local/lib/R/site-library/duckdb/libs/duckdb.so: cannot open shared object file: No such file or directory
> q()

❯ ls /usr/local/lib/R/site-library/duckdb/libs
duckdb.so

As you can see, the duckdb.so file does exist at the expected location.

To Reproduce

docker run -it vnijs/rsm-jupyter:2.5.0 /usr/local/bin/R

then use library(duckdb)

Docker file and set up: https://github.com/radiant-rstats/docker/blob/master/rsm-jupyter/Dockerfile

OS:

Ubuntu 22.04 Docker

DuckDB Version:

0.4.0

DuckDB Client:

R

Full Name:

Vincent Nijs (https://rady.ucsd.edu/faculty-research/faculty/vincent-nijs.html)

Affiliation:

Rady School of Management @ UCSD

Have you tried this on the latest `master` branch?

I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

I agree

The text was updated successfully, but these errors were encountered:

hannes · 2022-08-30T04:59:43Z

Does the Makevars workaround in the lined issue work for you?

vnijs · 2022-08-30T22:18:55Z

Thanks for the response @hannes. I didn't see anything in the posted issues related to this so I take it this issue is new.

Honestly, I'd prefer to avoid messing with the R build configuration but if that is the best option, I'll give it a try.

I'd also really like to understand why I'm getting this message after installing the binary package on aarch64.

hannes · 2022-08-31T04:05:32Z

If you read the post you will see that there is not much we can do there, sorry

vnijs · 2022-08-31T04:34:25Z

The issue I posted was really about installing binary packages from CRAN using the below in a Docker. The install install seems to succeed.

RUN R -e "options(HTTPUserAgent = sprintf('R/%s R (%s)', getRversion(), paste(getRversion(), R.version['platform'], R.version['arch'], R.version['os']))); install.packages(c('arrow', 'duckdb'), repos = 'https://packagemanager.rstudio.com/all/__linux__/jammy/latest', Ncpus=-1)"

However, when trying to load the library I get the following error message.

> library(duckdb)
Loading required package: DBI
Error: package or namespace load failed for 'duckdb' in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so':
  /usr/local/lib/R/site-library/duckdb/libs/duckdb.so: cannot open shared object file: No such file or directory
> q()

❯ ls /usr/local/lib/R/site-library/duckdb/libs
duckdb.so

That said, the workaround for installing from source did work so I have duckdb running on Ubuntu 22.04 on aarch64.

hannes · 2022-08-31T04:52:20Z

Great to hear you got it running.

benz0li · 2022-09-02T08:35:29Z

@vnijs FYI

[...] RStudio Package Manager (RSPM) does not provide R binary packages for arm64 [...]

Originally posted by @eitsupi in rocker-org/rocker-versioned2#144 (comment)

eitsupi · 2022-09-02T10:33:24Z

Yeah, inexplicably, when we try to install binary packages from RSPM on an arm64 machine, we get amd64 binaries. (Presumably RSPM doesn't care about the architecture of the client at this point)
rocker-org/rocker-versioned2#195

vnijs · 2022-09-02T16:59:59Z

@benz0li and @eitsupi Thanks for the update! That is unfortunate but helps me understand the issue.

Question: Have you by any chance been able to install arrow on aarch64? I'm having a lot of issues with that similar to issues you identified for duckdb. See also issue posted below.

https://issues.apache.org/jira/projects/ARROW/issues/ARROW-17374?filter=allopenissues

eitsupi · 2022-09-02T17:12:57Z

I haven't built in over 6 months because duckdb started causing builds to fail, but at least as of 6 months ago, it was possible to install arrow.
https://github.com/eitsupi/r-ver

$ docker run --rm -it ghcr.io/eitsupi/r-ver/tidyverse:latest@sha256:fd1f8dc26a6b652e376e16cb6e06361696828e
ad3573633a59e8f2952440a881 R
Unable to find image 'ghcr.io/eitsupi/r-ver/tidyverse:latest@sha256:fd1f8dc26a6b652e376e16cb6e06361696828ead3573633a59e8f2952440a881' locally
ghcr.io/eitsupi/r-ver/tidyverse@sha256:fd1f8dc26a6b652e376e16cb6e06361696828ead3573633a59e8f2952440a881: Pulling from eitsupi/r-ver/tidyverse
5f3d23ccb99f: Already exists
720d6608c4f9: Already exists
d3c9ca5a5443: Already exists
650fc0d1c376: Pull complete
010f88a1eccd: Pull complete
a3c81fcc4b72: Pull complete
d6cff8994a19: Pull complete
611ea251ed7d: Pull complete
7fcc5bd84607: Pull complete
Digest: sha256:fd1f8dc26a6b652e376e16cb6e06361696828ead3573633a59e8f2952440a881
Status: Downloaded newer image for ghcr.io/eitsupi/r-ver/tidyverse@sha256:fd1f8dc26a6b652e376e16cb6e06361696828ead3573633a59e8f2952440a881

R version 4.1.2 (2021-11-01) -- "Bird Hippie"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> arrow::arrow_info()
Arrow package version: 6.0.1

Capabilities:

dataset    TRUE
parquet    TRUE
json       TRUE
s3        FALSE
utf8proc   TRUE
re2        TRUE
snappy    FALSE
gzip      FALSE
brotli    FALSE
zstd      FALSE
lz4       FALSE
lz4_frame FALSE
lzo       FALSE
bz2       FALSE
jemalloc  FALSE
mimalloc  FALSE

To reinstall with more optional capabilities enabled, see
   https://arrow.apache.org/docs/r/articles/install.html

Memory:

Allocator  system
Current   0 bytes
Max       0 bytes

Runtime:

SIMD Level          none
Detected SIMD Level none

Build:

C++ Library Version  6.0.1
C++ Compiler           GNU
C++ Compiler Version 9.3.0

However, due to emulation by qemu, the build was very slow and it took about 3 hours to install just the arrow package......

benz0li · 2022-09-02T17:44:50Z

Question: Have you by any chance been able to install arrow on aarch64?

Yes. See

Dockerfile: https://gitlab.b-data.ch/r/tidyverse/-/blob/main/tidyverse/latest.Dockerfile
Build log: https://gitlab.b-data.ch/r/tidyverse/-/jobs/32936

docker run --rm registry.gitlab.b-data.ch/r/tidyverse R -e 'arrow::arrow_info()'

R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> arrow::arrow_info()
Arrow package version: 9.0.0

Capabilities:
               
dataset    TRUE
substrait FALSE
parquet    TRUE
json       TRUE
s3        FALSE
gcs       FALSE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip      FALSE
brotli    FALSE
zstd      FALSE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2       FALSE
jemalloc  FALSE
mimalloc  FALSE

To reinstall with more optional capabilities enabled, see
   https://arrow.apache.org/docs/r/articles/install.html

Memory:
                 
Allocator  system
Current   0 bytes
Max       0 bytes

Runtime:
                        
SIMD Level          none
Detected SIMD Level none

Build:
                           
C++ Library Version   9.0.0
C++ Compiler            GNU
C++ Compiler Version 10.2.1

ℹ️ Built on debian:bullseye, using a customised and patched R installation from source (*PICFLAGS = -fPIC for aarch64, stripped).

👉 Packages are also built from source and stripped as _R_SHLIB_STRIP_=true is set in /usr/local/lib/R/etc/Renviron.site.

hannes closed this as completed Aug 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so': #4532

unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so': #4532

vnijs commented Aug 29, 2022 •

edited

hannes commented Aug 30, 2022

vnijs commented Aug 30, 2022

hannes commented Aug 31, 2022

vnijs commented Aug 31, 2022

hannes commented Aug 31, 2022

benz0li commented Sep 2, 2022

eitsupi commented Sep 2, 2022 •

edited

vnijs commented Sep 2, 2022

eitsupi commented Sep 2, 2022

benz0li commented Sep 2, 2022 •

edited

unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so': #4532

unable to load shared object '/usr/local/lib/R/site-library/duckdb/libs/duckdb.so': #4532

Comments

vnijs commented Aug 29, 2022 • edited

What happens?

To Reproduce

OS:

DuckDB Version:

DuckDB Client:

Full Name:

Affiliation:

Have you tried this on the latest master branch?

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

hannes commented Aug 30, 2022

vnijs commented Aug 30, 2022

hannes commented Aug 31, 2022

vnijs commented Aug 31, 2022

hannes commented Aug 31, 2022

benz0li commented Sep 2, 2022

eitsupi commented Sep 2, 2022 • edited

vnijs commented Sep 2, 2022

eitsupi commented Sep 2, 2022

benz0li commented Sep 2, 2022 • edited

vnijs commented Aug 29, 2022 •

edited

Have you tried this on the latest `master` branch?

eitsupi commented Sep 2, 2022 •

edited

benz0li commented Sep 2, 2022 •

edited