Error loading ROracle after upgrading to RStudio 1.2.5001

[wkdavis]

System details

RStudio Edition : Desktop
RStudio Version : 1.2.5001
OS Version      : MacOS Catalina
R Version       : 3.6.1

Steps to reproduce the problem

1. Install Oracle Instant Client (version 12.2.0.1.0).
   • Install Basic package to /opt/ora12/instantclient_12_2
   • Install SDK to /opt/ora12/instantclient_12_2 (will create /opt/ora12/instantclient_12_2/sdk and put content there).
2. Create dynamic links for instant client

ln -s /opt/ora12/instantclient_12_2/libclntsh.dylib.12.1 /opt/ora12/instantclient_12_2/libclntsh.dylib
ln -s /opt/ora12/instantclient_12_2/libocci.dylib.12.1 /opt/ora12/instantclient_12_2/libocci.dylib

3. Create dynamic links for R

ln -s /Library/Frameworks/R.framework/Resources/lib/libclntsh.dylib.12.1 /Library/Frameworks/R.framework/Resources/lib/libclntsh.dylib
ln -s /opt/ora12/instantclient_12_2/libocci.dylib /Library/Frameworks/R.framework/Resources/lib/libclntsh.dylib.12.1

4. Update ~/.Renviron:

OCI_LIB=/opt/ora12/instantclient_12_2
OCI_INC=/opt/ora12/instantclient_12_2/sdk/include

5. Install ROracle (from source) install.packages("ROracle")
6. Load ROracle

Describe the problem in detail

Loading ROracle results in an error saying that an image can't be found.

> library(ROracle)
Error: package or namespace load failed for ‘ROracle’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Rlibs/ROracle/libs/ROracle.so':
  dlopen(/Library/Frameworks/R.framework/Rlibs/ROracle/libs/ROracle.so, 6): Library not loaded: @rpath/libclntsh.dylib.12.1
  Referenced from: /Library/Frameworks/R.framework/Rlibs/ROracle/libs/ROracle.so
  Reason: image not found

Other info:

> Sys.info()
sysname 
"Darwin" 
release 
19.0.0" 
version 
"Darwin Kernel Version 19.0.0: Wed Sep 25 20:18:50 PDT 2019; root:xnu-6153.11.26~2/RELEASE_X86_64" 
nodename 
"wkdavis.local" 
machine 
"x86_64" 
login 
"wkdavis" 
user 
"wkdavis" 
effective_user 
"wkdavis" 

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RJDBC_0.2-7.1 rJava_0.9-11  DBI_1.0.0    

loaded via a namespace (and not attached):
[1] compiler_3.6.1 tools_3.6.1   

Describe the behavior you expected

I expected the ROracle package to load correctly. Note that when I revert to RStudio 1.2.1335 the error does not occur and ROracle loads correctly.

[ronblum]

@wkdavis Thank you for raising the issue. Do you have multiple versions of R sharing a common library path? This is probably a a duplicate of #5238 and #5442, as I am able to cause a problem by setting R_LIBS to a common directory, installing ROracle in 3.5.3, then switching to 3.6.1.

[wkdavis]

That might have something to do with it. I do have a common directory R_LIBS=/Library/Frameworks/R.framework/Rlibs. Similarly to #5238, if I try to run library(glue) I get the error

Library not loaded: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libR.dylib

which references my old 3.5 installation. However, what is odd is that I still get the ROracle error even when I install ROracle with 3.6.1, and the error message changes to reference @rpath/libclntsh.dylib.12.1 instead of a specific version of R. I'm not familiar with the @rpath syntax.

[wkdavis]

I tried removing the common library and installed in

> .libPaths()
[1] "/Library/Frameworks/R.framework/Versions/3.6/Resources/library"

As before, ROracle installs correctly but errors on load:

> library(ROracle)
Loading required package: DBI
Error: package or namespace load failed for ‘ROracle’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so, 6): Library not loaded: @rpath/libclntsh.dylib.12.1
  Referenced from: /Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so
  Reason: image not found

with the same reference to @rpath/libclntsh.dylib.12.1. However, library(glue) now loads correctly, so it seems to be isolated to the combination of ROracle and RStudio 1.2.5001.

[kevinushey]

I suspect this is another consequence of the hardened runtime requirements for RStudio on macOS.

You can work around this with install_name_tool. In particular, we want to re-write the path in which that library searches for libclntsh.dylib. The following works for me:

# NOTE: set up to your own installation path as appropriate
Sys.setenv(
  OCI_LIB = "/opt/local/oracle/instantclient/19_3",
  OCI_INC = "/opt/local/oracle/instantclient/19_3/sdk/include"
)

# loading will fail after install, so disable load
install.packages("ROracle", INSTALL_opts = "--no-test-load")

# patch ROracle.so
so <- system.file("libs/ROracle.so", package = "ROracle")

args <- c(
  "-change",
  "@rpath/libclntsh.dylib.19.1",
  "/opt/local/oracle/instantclient/19_3/libclntsh.dylib.19.1",
  so
)

system2("install_name_tool", args)

# can now succesfully load ROracle
library(ROracle)          

Let me know if this is sufficient.

[wkdavis]

That worked, thank you!

Something interesting I noticed yesterday is that I installed ROracle (without the information in your fix and without the INSTALL_opts = "--no-test-load" argument) and it installed without error, including testing if it could be loaded:

> install.packages("ROracle")
Warning in install.packages :
  unable to access index for repository ...:
  cannot open URL ...
Package which is only available in source form, and may need compilation of C/C++/Fortran: ‘ROracle’
Do you want to attempt to install these from sources? (Yes/no/cancel) yes
installing the source package ‘ROracle’

trying URL 'https://cran.rstudio.com/src/contrib/ROracle_1.3-1.tar.gz'
Content type 'application/x-gzip' length 308252 bytes (301 KB)
==================================================
downloaded 301 KB

* installing *source* package ‘ROracle’ ...
** package ‘ROracle’ successfully unpacked and MD5 sums checked
** using staged installation
configure: creating ./config.status
config.status: creating src/Makevars
** libs
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/ora12/instantclient_12_2/sdk/include  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include  -fPIC  -Wall -g -O2  -c rodbi.c -o rodbi.o
rodbi.c:2432:53: warning: format specifies type 'long long' but the argument has type 'sb8' (aka 'long') [-Wformat]
        sprintf(err_buf, RODBI_ERR_BIND_VAL_TOOBIG, bndsz);
                         ~~~~~~~~~~~~~~~~~~~~~~~~~  ^~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:47:56: note: expanded from macro 'sprintf'
  __builtin___sprintf_chk (str, 0, __darwin_obsz(str), __VA_ARGS__)
                                                       ^~~~~~~~~~~
rodbi.c:2469:53: warning: format specifies type 'long long' but the argument has type 'sb8' (aka 'long') [-Wformat]
        sprintf(err_buf, RODBI_ERR_BIND_VAL_TOOBIG, bndsz);
                         ~~~~~~~~~~~~~~~~~~~~~~~~~  ^~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:47:56: note: expanded from macro 'sprintf'
  __builtin___sprintf_chk (str, 0, __darwin_obsz(str), __VA_ARGS__)
                                                       ^~~~~~~~~~~
rodbi.c:2816:55: warning: format specifies type 'long long' but the argument has type 'sb8' (aka 'long') [-Wformat]
          sprintf(err_buf, RODBI_ERR_BIND_VAL_TOOBIG, bndsz);
                           ~~~~~~~~~~~~~~~~~~~~~~~~~  ^~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:47:56: note: expanded from macro 'sprintf'
  __builtin___sprintf_chk (str, 0, __darwin_obsz(str), __VA_ARGS__)
                                                       ^~~~~~~~~~~
rodbi.c:2861:55: warning: format specifies type 'long long' but the argument has type 'sb8' (aka 'long') [-Wformat]
          sprintf(err_buf, RODBI_ERR_BIND_VAL_TOOBIG, bndsz);
                           ~~~~~~~~~~~~~~~~~~~~~~~~~  ^~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h:47:56: note: expanded from macro 'sprintf'
  __builtin___sprintf_chk (str, 0, __darwin_obsz(str), __VA_ARGS__)
                                                       ^~~~~~~~~~~
4 warnings generated.
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/ora12/instantclient_12_2/sdk/include  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include  -fPIC  -Wall -g -O2  -c rooci.c -o rooci.o
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o ROracle.so rodbi.o rooci.o -L/opt/ora12/instantclient_12_2 -lclntsh -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.6/Resources/library/00LOCK-ROracle/00new/ROracle/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (ROracle)

But then I would get the error when I tried to load it:

> library(ROracle)
Error: package or namespace load failed for ‘ROracle’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so, 6): Library not loaded: @rpath/libclntsh.dylib.12.1
  Referenced from: /Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so
  Reason: image not found

[kevinushey]

It sounds like the underlying issue is now understood and solved, so I believe this can be closed.

[wkdavis]

@kevinushey I'm okay with closing it, but why would the package install successfully, including

** testing if installed package can be loaded from final location

but then fail to load when I call library(ROracle)?

[kevinushey]

Honestly, I'm not exactly sure. All I know is, based on the error message:

> library(ROracle)
Error: package or namespace load failed for ‘ROracle’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so, 6): Library not loaded: @rpath/libclntsh.dylib.12.1
  Referenced from: /Library/Frameworks/R.framework/Versions/3.6/Resources/library/ROracle/libs/ROracle.so
  Reason: image not found

it's because libclntsh.dylib.12.1 could not be found using the @rpath. See https://medium.com/@donblas/fun-with-rpath-otool-and-install-name-tool-e3e41ae86172 for some more details; install_name_tool would still be the workaround to make sure the library is found in the right place.

[wkdavis]

Okay no problem, thanks for the follow-up!