UndefinedBehaviorSanitizer: object of type SharedMemoryBigMatrix is not BigMatrix

[rdrr1990]

Hi @kaneplusplus ,

I use library(bigmemory) in my package bigKRLS (with @rbshaffer) (and it's been highly effective, increasing what a typical laptop can do fivefold). bigKRLS also uses Rcpp and RcppArmadillo. bigKRLSuseslibrary(parallel) for the marginal effects. (bigKRLSregressesyon some matrixX. If ncol(X) = p, bigKRLSusesporNcores - 2`, whichever is less).

When I submit to CRAN, I get the following error:

> N <- 500  # proceed with caution above N = 5,000 for system with 8 gigs made available to R
> P <- 4
> X <- matrix(rnorm(N*P), ncol=P)
> b <- 1:P 
> y <- sin(X[,1]) + X %*% b + rnorm(N)
> out <- bigKRLS(y, X, Ncores=1)
gauss_kernel.cpp:38:40: runtime error: member call on address 0x6120001ac8c0 which does not point to an object of type 'BigMatrix'
0x6120001ac8c0: note: object is of type 'SharedMemoryBigMatrix'
 25 00 80 47  38 80 92 67 87 7f 00 00  04 00 00 00 00 00 00 00  f4 01 00 00 00 00 00 00  f4 01 00 00

Similar errors repeat for the other function calls.

This means that X is coerced to a big.matrix by as.big.matrix() without issue but then by the time the Rcpp function is called there is apparently some slippage between SharedBigMatrix and BigMatrix. Any thoughts or suggestions on how to address this issue? (Also, please let me know if more detail re: bigKRLS would be helpful.)

[kaneplusplus]

Hi @rdrr1990. I vaguely remember a problem with the clang sanitizer options not allowing inheritance options, which are relevant when a SharedBigMatrix type needs methods from the BigMatrix type.

I'm over-tasked for the rest of the week. I'll take a look this weekend. If I don't update this thread on Monday would you mind pinging me?

Thanks very much.

[privefl]

Are you coercing X to a big.matrix in bigKRLS?

I recently decided to use only filebacked memory files.
Maybe using a temporary filebacked big.matrix would solve your problem and be as efficient.

E.g., make an utility function like this:

as.FBM <- function(X) {
  tmp <- tempfile()
  as.big.matrix(X, backingfile = basename(tmp), 
                backingpath = dirname(tmp),
                descriptorfile = paste0(basename(tmp), ".desc"))
}

[rdrr1990]

Thanks for the suggestion, @privefl . bigKRLS currently uses filebacked big.matrix objects when the user uses multiple cores. I will take some time this week to see if using them more systematically helps. Do you happen to know whether there's a setting (or something) that makes R display C errors like that (currently we only see the output when we submit to CRAN)?

[privefl]

I don't know. I had problems with parallelism on CRAN.
If it is only for examples and tests, I suggest skipping them on CRAN.

[rdrr1990]

We did as well, which is why the example calls bigKRLS(y, X, Ncores=1) instead of bigKRLS(y, X). Disabling the examples is not a bad idea though.

[jaredhuling]

I'm having this issue as well, but

runtime error: member call on address ... which does not point to an object of type 'BigMatrix'
...: note: object is of type 'FileBackedBigMatrix'

[kaneplusplus]

I'm not sure. As mentioned before I've only seen this when the clang sanitizer was set to not allow inheritance. Note that a FileBackedBigMatrix inherits from a SharedBigMatrix, which inherits from a BigMatrix. See here for the declarations.