Intensive calling for R functions causes crash

[renkun-ken]

Intensive calling for R functions like lm() is highly probable to cause crash. The code may look like this:

open RProvider
open RProvider.``base``
open RProvider.stats
open RProvider.fUnitRoots

let s =
    [|1..2000|]
    |> Array.map (fun i ->
        let x = R.rnorm(100)
        let y = R.rnorm(100)
        R.assign("x",x) |> ignore
        R.assign("y",y) |> ignore
        let m = R.lm("y~x") // It almost always crashes at this line.
        let res = m.AsList().["residuals"].AsNumeric()
        let test1 = R.adfTest(res)
        let p:double = test1.GetAttribute("test").AsList().["p.value"].AsNumeric().[0]
        p
    )

When the program is running, it almost always crashes for the following exception:

An unhandled exception of type 'System.AccessViolationException' occurred in RDotNet.dll

Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

However, when I directly use RDotNet to implement exactly the same thing, no error occurs at all. Actually, I "solved" the problem by adding a GC.Collect() in each iteration, and the probability of error is hugely reduced. I wonder what's the root of the error?

[hmansell]

It's likely some kind of subtle bug in R.NET. We don't do any unmanaged memory management in RProvider. Presumably you are using it in some way that looks the same but actually is subtly different. I fixed some bugs in SymbolicExpression resource management before, but I am sure some remain. It's very difficult to do the interop with unmanaged code correctly.

When I get a chance, I'll try to take a look. Having a robust repro helps!

[renkun-ken]

I raised an issue in RDotNet project (https://rdotnet.codeplex.com/workitem/67) and they redirect me to Issue #4. Here I provide a minimal reproducible example where RProvider causes an error while RDotNet does not.

Using RDotNet:

open RDotNet
open RDotNet.NativeLibrary
open RDotNet.Internals

let REngine = REngine.CreateInstance("R_INSTANCE", @"C:\Program Files\R\R-3.0.2\bin\x64\R.dll")
let R (statement:string) = REngine.Evaluate statement
REngine.Initialize()

[<EntryPoint>]
let main argv = 
    let beta = 
        [ for i in 1..1000 do
              let x = R "x <- rnorm(1000)"
              let y = R "y <- rnorm(1000)"
              let m = R "m <- lm(y~x)"
              let beta = (R "m$coefficients[2]").AsNumeric().[0]
              printfn "%f" beta
              yield beta ]

    let meanBeta = beta |> List.average
    0 // return an integer exit code

This code runs without any error.

RProvider:

open RDotNet
open RProvider
open RProvider.``base``
open RProvider.stats

[<EntryPoint>]
let main argv =
    let beta =
        [
            for i in 1..1000 do
                let x = R.rnorm(1000)
                let y = R.rnorm(1000)
                let df = R.data_frame(namedParams [ "y", box y; "x", box x; ])
                let m = R.lm(df)
                let beta = m.AsList().["coefficients"].AsNumeric().[1]
                printfn "%f" beta
                yield beta
        ]
    let meanBeta = beta |> List.average
    0

Every time I run the code above, at some i an error occurs at let df = R.data_frame or let m = R.lm(df):

An unhandled exception of type 'System.AccessViolationException' occurred in RDotNet.dll
Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

How do we debug the code and see what happens?

[tpetricek]

This will be fixed once we update to new version of R.NET & merge @dcharbon changes, so I'm closing this.