Update TBB interface and allow using external lib

[hsbadr]

This closes #107 and is related to stan-dev/math#2257. It adds support for the new interface of Intel TBB and allows using external library (e.g., with Intel OneAPI), using TBB_LIB and TBB_INC and environment variables. The updated TBB functionality are summarized here and the release notes and backward compatibility are reported here.

These changes have been successfully tested with both embedded TBB source code and TBB library from Intel OneAPI 2021.1.

• Added support for external TBB library via TBB_INC and/or TBB_LIB environment variables
• Updated TBB functionality for the new interface
• Falling back to building TBB from local source code
• Backward TBB compatibility based on __TBB_tbb_stddef_H
• Resolved conflicts between system and local TBB headers
• Updated CXXFLAGS and LDFLAGS
• Fixed URLs, used HTTPS, and minor cleanups
• Updated NEWS with incremented package version
• Updated package DESCRIPTION and bumped version
• Added contributor to package DESCRIPTION

[eddelbuettel]

Conceptually speaking, could you foresee a modification where both a vendored and a system library could be used? I contributed such a scheme once to another (smaller) package---in the R ecosystem we have our reasons for vendoring to ensure "easy" buildability for everybody across multiple operating systems and many different versions. Now, I have a lot of sympathy for the "system" view and yet while I am contributor to one of the Linux distros still "embed" libraries in R package "CRAN style" (and am currently doing such an upgrade for Boost 1.75.0 in the BH package I maintain). This a call to make for @kevinushey but I fear removing the embedded source may create problems for the package.

[hsbadr]

@eddelbuettel Agreed. I'm testing this right now because I have recognized that this fails if the environment is not set correctly (i.e., if TBB library is not found in the system or if the environment variable TBB_LIBRARY_FILE is not set). I will let it fall back to building TBB from the embedded source code.

[kevinushey]

Right, we need to keep the embedded code so this can still build on Windows + macOS. I agree that having RcppParallel use the system TBB library on Linux-alikes would be worth doing (and we should make it possible for the user to configure which version is loaded at runtime regardless).

[hsbadr]

This is not ready yet... need to fix the conflicts with system headers and test it.

[hsbadr]

@kevinushey This is ready to go. Everything has been summarized in the PR descriptions and the package NEWS. Users can opt in using the system TBB library or building it from the embedded source code at compilation time; header conflicts are automatically resolved. This should be it since runtime changes is tricky since the package shared library uses TBB headers and, with the new TBB interface that breaks backward compatibility, it isn't recommended to change the loaded TBB library without recompilation.

[kevinushey]

Thanks; I'll try to review this soon. (I see you've been making a couple more commits since you said this was ready for review so just want let the dust settle first)

[hsbadr]

@kevinushey Sounds good! It was ready, but then I've been fixing reverse dependencies for packages that link directly to the libs in
RcppParallel lib directory without using RcppParallel::LdFlags() and RcppParallel::CxxFlags() as well as handling odd cases where the system has multiple versions of TBB headers (e.g., old TBB library, libtbb-dev, is installed in /usr/ while a new library, OneAPI TBB, is also installed in /opt/intel/).

The conflicting headers actually break the current version of RcppParallel, especially if it unexpectedly uses the headers from the new library. It works fine for me now in all situations, but I'll keep it updated if I run into any issues.

[kevinushey]

Overall this looks good to me -- thank you for putting this together! Some questions I have:

If RcppParallel is built using the system version of TBB, should it "remember" the TBB include / library paths used at build time? Or is it okay to continue respecting TBB_LIB and TBB_INC at runtime, even if it differs? (What happens if TBB_LIB + TBB_INC are set at build time, but are not set at runtime -- can the package still run?)

What's the right way for users installing RcppParallel to use the system copy of TBB? Does it suffice to just set the TBB_INC and TBB_LIB environment variables, or do they have to be passed as configure variables?

[hsbadr]

If RcppParallel is built using the system version of TBB, should it "remember" the TBB include / library paths used at build time? Or is it okay to continue respecting TBB_LIB and TBB_INC at runtime, even if it differs?

RcppParallel will be successfully loaded at runtime if the TBB libs are in the system regardless of the environment variables. This is better than caching the paths as the users may reinstall the external libs somewhere else in the system (e.g., in /opt vs /usr), but the package should be recompiled if the system TBB changes the interface, which will cause undefined references when loading the new TBB libs.

What happens if TBB_LIB + TBB_INC are set at build time, but are not set at runtime -- can the package still run?

Yes. Though, the environment variables are required to report the correct RcppParallel::LdFlags() and RcppParallel::CxxFlags(). I've tested this right now and the packages runs while the flags functions generate the following warnings:

Warning messages:
1: In fun(libname, pkgname) : TBB library  not found.
2: In fun(libname, pkgname) : TBB malloc library  not found.

So, it's recommend to keep the environment variables defined at runtime. They can be updated to use different paths too, if the different TBB versions are compatible (use the same interface).

But, we already copy the libs inside RcppParallel installation directory and could add the headers too and rely on those in the runtime, If you think this will be better.

What's the right way for users installing RcppParallel to use the system copy of TBB? Does it suffice to just set the TBB_INC and TBB_LIB environment variables, or do they have to be passed as configure variables?

The users need to set the environment variables (specifically: TBB for the installation prefix, TBB_INC for the directory that includes the header files, and TBB_LIB for the libraries directory). For example, this can be automatically set via source /opt/intel/oneapi/setvars.sh if Intel OneAPI is installed or

export TBB="/opt/intel/oneapi/tbb/latest"
export TBB_INC="/opt/intel/oneapi/tbb/latest/include"
export TBB_LIB="/opt/intel/oneapi/tbb/latest/lib/intel64/gcc4.8"

[hsbadr]

@kevinushey The warnings I mentioned above are from tbbLibPath() when loading the package if external TBB libs are linked at build time but the environment variables aren't set at runtime. This is basically because the new TBB libs I'm using have a different suffix (libtbb.s.12 vs libtbb.so.2). I'll fix this and the package should run anyway.

[hsbadr]

@kevinushey With the above commits, the package should run in all cases.

[kevinushey]

Awesome; thank you very much!