very weird bug in library.dynam2 when interaction with DLL and RJSONIO import

[kforner]

Hello,

I've painfully managed to narrow a long-standing bug affecting my code, and to make a reproducible example.

The Context:

• pkgdll: defines a dummy dll (not even called)
• rjsonioUser: imports 2 functions from RJSONIO
• these 2 packages are independent

The problem:
reloading pkgdll makes rjsonioUser crash.

This bug only happens in R >=3, not in R 2.15.3
and with the github master and CRAN version of devtoos, on linux and osX.

The reproducible example is here: https://github.com/kforner/devtools_dll_bug.git
There are scripts called bugN.R that exemplify the bug.
They do deeper and deeper into the problem, starting from load_all to finish with dyn.load(), which is an internal function.

Run them like: Rscript bug1.R

The error message is:
Error in .Call("R_fromJSON", content, as.integer(sum(simplify)), nullValue, : "R_fromJSON" not resolved from current namespace (RJSONIO) Calls: rjson ... fromJSON -> fromJSON -> fromJSON -> fromJSON -> .Call Execution halted

I do not know if it is related to RJSONIO.

I can try to help, but I'm currently stuck, don't know what to investigate.

Karl Forner

[hadley]

Thanks! I've had that problem too, but never managed to get to the source. Given that it only seems to occur with RJSONIO, I'd wonder if something there is set up incorrectly in the NAMESPACE or dynamic function registration.

[wch]

Does the same error happen when you do it the non-devtools way?

library(pkgdll)
library(rjsonioUser)

rjson(1:10)

detach('package:pkgdll', unload=TRUE)
# Or maybe this:
# unloadNamespace('pkgdll')

library(pkgdll)

rjson(1:10)

[kforner]

@hadley I tried to have a quick look at RJSONIO, but could not find anything special, thought I'm not very experienced with pkgs with dlls

@wch
nope.
I tried with detach or unloadNamespace, both work.

[kforner]

I read that jsonlite was a fork of RJSONIO. I pushed a new test (bug1a.R) and it does not crash at all. So there seems indeed to be a very nasty business with RJSONIO.

[hadley]

I've basically abandoned RSJONIO because of this problem. It doesn't seem worth the effort to figure out exactly what's going wrong.

[kforner]

but you still use it in shiny right ? 'cause we had the problem recently.

[hadley]

You'll have to take that up with the shiny maintainers ;)

[wch]

I did a bit more investigating of this issue. The reason it doesn't cause a problem with unloadNamespace or detach is because those don't unload the pkgdll.so shared library.

First modify pkgdll to have an .onUnload function like this (which is considered good practice; see http://r.789695.n4.nabble.com/Using-onUnload-to-unload-DLLs-td4637562.html):

.onUnload <- function (libpath) {
  library.dynam.unload("pkgdll", libpath)
}

Then install pkgdll, and run the following. You'll get the same error:

library(devtools)
library(methods)

install('pkgdll')
library(pkgdll)
load_all('rjsonioUser/')

rjson(1:10)
cat('######################## after first call\n')

unloadNamespace('pkgdll')
library(pkgdll)

cat('=============================before call2\n')
rjson(1:10)
# Error in .Call("R_fromJSON", content, as.integer(sum(simplify)), nullValue,  : 
#  "R_fromJSON" not resolved from current namespace (RJSONIO) 
cat('#############################after call2\n')

I suspect it has something to do with R trying to unload a DLL that wasn't the last one loaded. If, after loading the packages, you run .dynLibs(), you'll see that the RJSONIO DLL was loaded after pkgdll.

If you want to trace into the problem, you can do: debug(shiny:::library.dynam2) and run your original bug1.R code. Step through this function until it gets to the line:

dllinfo <- dyn.load(dllfile)

Before you run that line, you can do rjson(1:10). After you run that line, you'll get an error.

In short, it looks like the problem is some interaction between R and RJSONIO. The reason you don't get it when you do unloadNamespace('pkgdll') is because that won't unload the DLL (unless you add an .onUnload function as I wrote above). Devtools tries harder than unloadNamespace to really unload a DLL, which is why you see the issue when you use devtools.