Optional use of tbbmalloc_proxy (to speedup macOS)?

[wds15]

The RcppParallel provided Intel TBB library is being used in RStan for a while now. In benchmarks we found out that specifically on macOS the use of the tbbmalloc_proxy library speeds up Stan programs by ~20%. Loading the tbbmalloc_proxy library replaces all calls to the system malloc to the replacement from the TBB. The upside of doing so is that no source code needs to be changed at all in order to gain from the speed benefits of the TBB provided malloc, which is designed to work well will threaded programs (see here for details).

From running benchmarks with Stan programs it turned out that there are clear speedups of ~20% on macOS when using the TBB malloc while other platforms did not really gain in speed. This is why the TBB malloc is only enabled for Stan programs on macOS, but there the speed gains are really nice.

So I wonder if this would be of interest to enable (maybe optionally) loading the tbbmalloc_proxy with RcppParallel.

If that is an option, I am happy to provide a PR for such an optional feature. If there is anything to consider for such a feature, please let me know.

[kevinushey]

Sorry for taking so long to respond. I think we'd consider it, but I think it would have to be something that is opt-in (not on by default). Mainly because last time I tried, I saw issues with R hanging in R CMD check (evidently having trouble unloading the associated library).

I'm also not sure if it's safe to dynamically load tbbmalloc_proxy -- shouldn't it normally be loaded when the process is started, either via LD_PRELOAD or (on macOS) DYLD_INSERT_LIBRARIES?

[wds15]

Oh...a late answer is much better than no answer!

Of course, the use of this functionality should be optional, absolutely.

I wasn't aware that tbbmalloc_proxy was triggering the R CMD check hangs, which I also saw happening. Interesting.

In any case, I can report you that rstan programs run a lot faster once tbbmalloc_proxy is loaded on macOS.

So let's have a try is what I would say then.

[kevinushey]

Now implemented in the development version. From NEWS:

• The RCPP_PARALLEL_USE_TBBMALLOC_PROXY environment variable can now be used
  to control whether RcppParallel loads the tbbmalloc_proxy library on load.
  See https://www.threadingbuildingblocks.org/docs/help/tbb_userguide/Automically_Replacing_malloc.html
  for more information.

If you have a chance, please test it out and let me know what you find.

[wds15]

Whow! That was a quick turn around. Will test, sure.

[kevinushey]

Here's what I see with LLDB:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x1012f5140)
  * frame #0: 0x00007fff20427541 libsystem_malloc.dylib`default_zone_malloc + 22
    frame #1: 0x00007fff20441dfe libsystem_malloc.dylib`_malloc_zone_malloc + 118
    frame #2: 0x00000001006d9533 libR.dylib`R_AllocStringBuffer(blen=512, buf=0x00007fbc684d7ff0) at memory.c:4509:23 [opt]
    frame #3: 0x000000010063d4b9 libR.dylib`print2buff(strng="", d=0x00007fbc684d7fd0) at deparse.c:1480:5 [opt]
    frame #4: 0x000000010063bdbb libR.dylib`deparse1WithCutoff(call=0x00007fbc37ce2ee8, abbrev=FALSE, cutoff=<unavailable>, backtick=<unavailable>, opts=1089, nlines=0) at deparse.c:254:5 [opt]
    frame #5: 0x000000010063c1dc libR.dylib`Rf_deparse1m(call=0x00007fbc37ce2ee8, abbrev=FALSE, opts=1089) at deparse.c:197:19 [opt]
    frame #6: 0x0000000100672778 libR.dylib`R_GetTraceback(skip=<unavailable>) at errors.c:1514:19 [opt]
    frame #7: 0x00000001006ce4ae libR.dylib`sigactionSegv(signum=11, ip=0x00007fbc684d8580, context=<unavailable>) at main.c:603:2 [opt]
    frame #8: 0x00007fff20645d7d libsystem_platform.dylib`_sigtramp + 29
    frame #9: 0x00007fff20428125 libsystem_malloc.dylib`free + 105
    frame #10: 0x000000010c255eb6 dyld`ImageLoader::~ImageLoader() + 68
    frame #11: 0x000000010c263e21 dyld`ImageLoaderMachOCompressed::~ImageLoaderMachOCompressed() + 33
    frame #12: 0x000000010c2485b5 dyld`dyld::garbageCollectImages() + 964
    frame #13: 0x000000010c2537af dyld`dlclose + 191
    frame #14: 0x00007fff20619ab7 libdyld.dylib`dlclose + 170
    frame #15: 0x00000001005c3b70 libR.dylib`DeleteDLL(path=<unavailable>) at Rdynload.c:523:5 [opt]
    frame #16: 0x00000001005c3273 libR.dylib`AddDLL(path="/Users/kevinushey/Library/R/4.0/library/RcppParallel/lib/libtbbmalloc_proxy.dylib", asLocal=0, now=1, DLLsearchpath="") at Rdynload.c:583:5 [opt]

So the mismatch between allocators when loading + unloading the library seems to be causing trouble.

[wds15]

This is weird. It used to work with rstan 2.19 under R 3.6 for sure where I did load the tbbmalloc_proxy during rstan startup.

If I now use the dev version and try to load the tbbmalloc_proxy then I am getting a "memory not mapped" segfault once I load rstan. One can also reverse the order to load RcppParallel (without the proxy thing), then load rstan (which works then) and then load the proxy library. The result is again a segfault with "memory not mapped".

Let's ping @hsbadr ... maybe he has an idea on what to do?

(The proxy thing is used under CmdStan on macOS)

[kevinushey]

It probably works if you launch R with something like:

DYLD_INSERT_LIBRARIES=/path/to/tbbmalloc_proxy.dylib R

That is, you have to force R to use allocators defined by tbbmalloc_proxy on startup; trying to switch the active allocators at runtime is more likely to lead to trouble.

[hsbadr]

@wds15 Please try @kevinushey's suggestion. If it doesn't work, try to use an external TBB library for both RcppParallel and rstan, if possible.

[wds15]

Sounds like you guys are right. Here is what worked:

DYLD_LIBRARY_PATH=~/R/2020-06-17-transient/RcppParallel/lib/ DYLD_INSERT_LIBRARIES=~/R/2020-06-17-transient/RcppParallel/lib/libtbbmalloc_proxy.dylib Rscript test-rcppparallel.R

So I had to add the library path in addition as the proxy library loads the libtbbmalloc library as well. Doing so reduced the test program runtime from 99s to 91s - which I find quite nice given that nothing else has changed here.

Is there a way to make this functionality available to users in a not too hard way? Would the R startup process allow this in some way?

Maybe other non-Stan things speedup from this as well?

[kevinushey]

Is there a way to make this functionality available to users in a not too hard way? Would the R startup process allow this in some way?

As far as I know, not really. You could define your own R wrapper script that invoked R in that way with those environment variables set, I suppose?