Unexpected OpenMP in RcppArmadillo data structures?

[evanbiederstedt]

Hi @eddelbuettel CC @VPetukhov

Apologies if this is documented somewhere and we missed it.

We've been noticing some strange behavior with the following function jsDist(), which spins up multiple processes when run:

https://github.com/kharchenkolab/sccore/blob/master/src/matrix_utils.cpp#L6-L31

I cannot see why this would happen; my best guess currently is that something multithreaded in the RcppArmadillo data structures used within this function. Am I crazy thinking this?

Is there any way to conditionally disable/limit OpenMP for certain parts of RcppArmadillo?

Otherwise....I'm befuddled why this would be happening. We can disable OpenMP entirely of course in the Makevars files, but this doesn't explain why this function is multithreaded....

Thank you for the help, Evan

[eddelbuettel]

Is there any way to conditionally disable/limit OpenMP for certain parts of RcppArmadillo?

I supposed you could use the standard (OpenMP, MKL, ...) choice of limiting threads? I will take a quick look, Armadillo likely also has a toggle.

Now, can we call this "unexpected"? I am not sure given that each install.packages("RcppArmadillo") (or alike) shows (on a real operating system) the line

checking whether R CMD SHLIB can already compile programs using OpenMP... yes

[eddelbuettel]

Straight quote from the Armadillo documentation (and I apologise for the screenshot but copying and pasting brought html markup in):

With that, I think it might be appropriate to close this.

[evanbiederstedt]

Hi @eddelbuettel

Ok, this is clicking now. I think our assumption was that even with

checking whether R CMD SHLIB can already compile programs using OpenMP... yes

we would still need to manually set OpenMP....but it's in the Armadillo code.

With this, I think we can work on a solution: https://stackoverflow.com/questions/46723854/rcpparmadillo-using-autoconf-to-disable-openmp

I appreciate the help. @VPetukhov may have some more comments/questions here.

[eddelbuettel]

I actually really like Conrad's documentation in the one big page where, once you Ctrl-F for say "thread" as I did you also find entry points prohibiting it for particular functions. Maybe we can help some more?

And as to your edit right now: Indeed, for performance reasons, Armadillo allows itself to drop into OpenMP internally when it makes sense. So if you and your end also run OpenMP around it, bad things could happen.