Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update TBB interface and allow using external lib #2257

Merged
merged 5 commits into from
Dec 14, 2020

Conversation

hsbadr
Copy link
Member

@hsbadr hsbadr commented Dec 12, 2020

Summary

This adds support for the new interface of Intel TBB and allows using external library (e.g., with Intel OneAPI), using TBB_LIB and TBB_INC environment variables. To enable the new TBB interface, define TBB_INTERFACE_NEW in the user-defined variables in ~/.config/stan/make.local or make/local. The updated TBB functionality are summarized here and the release notes and backward compatibility are reported here.

Note that the internal TBB source code is ignored if TBB_LIB and TBB_INC environment variables are defined. TBB_INTERFACE_NEW can be set to true by default when the source code is updated. I'm not sure if you'd like to keep backward compatibility with the old TBB interface. If not, this new interface branching can be removed and fall back to the updated source code if the system headers are old (e.g., tbb/tbb_stddef.h exists). TBB_INTERFACE_VERSION is now moved to a new header tbb/version.h from tbb/tbb_stddef.h, which has been removed.

PS: Do not enable the new TBB interface with the current internal version of TBB, until it gets updated. Define TBB_INTERFACE_NEW, TBB_LIB and TBB_INC to use an updated external TBB library.

This is related to RcppCore/RcppParallel#141.

Tests

  • intel_tbb_new_init
  • intel_tbb_new_late_init

Side Effects

N/A

Release notes

Added support for the new TBB interface and allowed using an external TBB library.

Checklist

  • Math issue N/A

  • Copyright holder: Hamada S. Badr hamada.s.badr@gmail.com

  • the basic tests are passing

    • unit tests pass (to run, use: ./runTests.py test/unit)
    • header checks pass, (make test-headers)
    • dependencies checks pass, (make test-math-dependencies)
    • docs build, (make doxygen)
    • code passes the built in C++ standards checks (make cpplint)
  • the code is written in idiomatic C++ and changes are documented in the doxygen

  • the new changes are tested

hsbadr and others added 2 commits December 12, 2020 02:12
Define TBB_INTERFACE_NEW to use the new interface of Intel/OneAPI TBB system library.
This can be added in the user-defined variables in `~/.config/stan/make.local` or `make/local`.

TBB updated functionality: https://software.intel.com/content/www/us/en/develop/articles/tbb-revamp.html

TBB release notes and backward compatibility: https://software.intel.com/content/www/us/en/develop/articles/intel-oneapi-threading-building-blocks-release-notes.html

Intel OneAPI release notes: https://software.intel.com/content/www/us/en/develop/articles/intel-oneapi-toolkit-release-notes.html

Note that Intel OneAPI automatically sets the required environment variables for TBB.

Signed-off-by: Hamada S. Badr <hamada.s.badr@gmail.com>
Copy link
Member

@rok-cesnovar rok-cesnovar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @hsbadr! A couple of minor questions and comments.

test/unit/math/prim/core/init_threadpool_tbb_test.cpp Outdated Show resolved Hide resolved
test/unit/math/prim/core/init_threadpool_tbb_late_test.cpp Outdated Show resolved Hide resolved
stan/math/prim/core/init_threadpool_tbb.hpp Show resolved Hide resolved
make/compiler_flags Show resolved Hide resolved
@wds15
Copy link
Contributor

wds15 commented Dec 12, 2020

Also, do we need to branch in our sources on a macro like TBB_INTERFACE_NEW? I mean what is new and what old changes tomorrow. Instead I would prefer to branch based on a version number. The TBB is likely exposing some TBB version string with their headers in some of their defines. Couldn't we use that to branch?

@rok-cesnovar
Copy link
Member

Couldn't we use that to branch?

Good call. Agree.

@hsbadr
Copy link
Member Author

hsbadr commented Dec 12, 2020

Also, do we need to branch in our sources on a macro like TBB_INTERFACE_NEW? I mean what is new and what old changes tomorrow. Instead I would prefer to branch based on a version number. The TBB is likely exposing some TBB version string with their headers in some of their defines. Couldn't we use that to branch?

@wds15 @rok-cesnovar Good point. We can do it based on the version number, but TBB_INTERFACE_VERSION is now moved to a new header tbb/version.h from tbb/tbb_stddef.h, whcih has been removed. So, we may also use __TBB_tbb_stddef_H instead, which is defined in the old interface only or make use of __has_include. I didn't want to make invasive changes that you may not like. So, feel free to suggest changes that works for you.

@rok-cesnovar
Copy link
Member

Oh, that's unfortunate then.

I am leaning slightly towards __TBB_tbb_stddef_H with a comment of why that is there. I really hope we move to this new interface soon, we will also be able to solve #1949 in a simple way

@hsbadr
Copy link
Member Author

hsbadr commented Dec 12, 2020

The problem with using __TBB_tbb_stddef_H is that we will have to check if tbb/tbb_stddef.h exists and include it. I think it's cleaner to define the interface with the compiler flags in this case, which is exactly what TBB_INTERFACE_NEW does. Though, we should separate the new vs old interface from the system vs internal TBB linking. What about STAN_TBB = true for preferring the internal TBB source code? I'm trying to be consistent with the other configs such as STAN_OPENCL, but we could make it explicit, TBB_USE_SYSTEM_LIB, as you suggested.

@rok-cesnovar
Copy link
Member

Oh, but there is a slight difference there.

STAN_OPENCL=true means use OpenCL backend, STAN_MPI=true means use MPI backend and STAN_THREADS means use TBB for threading-enabled functions. This is why I don't feel like STAN_TBB is the best name here as it would suggest "use TBB" when it's actually "use the system installed TBB". TBB_USE_SYSTEM_LIB would be my choice, I am just not sure what to do for branching in the header file.

Let me take a look and get back to you tomorrow.

@hsbadr
Copy link
Member Author

hsbadr commented Dec 12, 2020

Now, TBB_USE_SYSTEM_LIB skips building the internal TBB source code and depends on the system variables, preferably set by Intel OneAPI. TBB_INTERFACE_NEW (or a renamed directive) can be set to true by default when the source code is updated. I'm not sure if you'd like to keep backward compatibility with the old TBB interface. If not, this branching can be removed and it falls back to the updated source code if the system headers are old (e.g., tbb/tbb_stddef.h exists).

@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.46 3.49 0.99 -0.84% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 0.99 -1.29% slower
eight_schools/eight_schools.stan 0.12 0.11 1.04 3.47% faster
gp_regr/gp_regr.stan 0.15 0.16 0.98 -1.78% slower
irt_2pl/irt_2pl.stan 5.61 5.44 1.03 2.91% faster
performance.compilation 88.78 85.74 1.04 3.42% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.42 8.4 1.0 0.16% faster
pkpd/one_comp_mm_elim_abs.stan 29.52 29.28 1.01 0.83% faster
sir/sir.stan 140.04 139.46 1.0 0.42% faster
gp_regr/gen_gp_data.stan 0.04 0.05 0.98 -2.02% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.96 2.91 1.02 1.66% faster
pkpd/sim_one_comp_mm_elim_abs.stan 0.38 0.38 1.0 0.04% faster
arK/arK.stan 2.47 2.49 0.99 -0.83% slower
arma/arma.stan 0.59 0.59 1.0 -0.12% slower
garch/garch.stan 0.62 0.6 1.03 3.26% faster
Mean result: 1.0065885829

Jenkins Console Log
Blue Ocean
Commit hash: 059775e


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

@wds15
Copy link
Contributor

wds15 commented Dec 13, 2020

Reading through this PR suggests that we do two things at the same time: external TBB and new TBB... but can this be separate?

I mean, maybe a user wants to use an old TBB of the system with stan-math or we may want to bump the TBB version in stan-math. Will these patches then still work as expected?

@hsbadr
Copy link
Member Author

hsbadr commented Dec 13, 2020

@wds15 Exactly! That's what the last commit does. And, yes, this will work in any combinations of TBB_INTERFACE_NEW (new vs old TBB interface) and/or TBB_USE_SYSTEM_LIB (system vs internal library). Though, the user is responsible for compatibility of the two options (i.e., the system TBB headers should be updated if both options are enabled). I tested this in multiple situations where the system TBB library uses the old interface (libtbb-dev on Ubuntu, TBB_INTERFACE_NEW not defined) and the beta version of TBB library in OneAPI 2021.1 (TBB_INTERFACE_NEW=true).

Here, if TBB_USE_SYSTEM_LIB is not defined, TBB_INTERFACE_NEW=true should be the default only when the internal TBB source code is updated to use the new interface.

@rok-cesnovar
Copy link
Member

This looks good to me, I would maybe change the INTERFACE NEW to TBB_USE_GLOBAL_CONTROL. After we switch to a TBB with this new interface this will be the default and only setting. @wds15 does that sound good for you?

@hsbadr
Copy link
Member Author

hsbadr commented Dec 13, 2020

@rok-cesnovar TBB_USE_GLOBAL_CONTROL would be a bit misleading since the old interface has tbb::global_control too. But, we could predefine __TBB_tbb_stddef_H in the compiler flags for the old interface and remove it when the internal TBB source code is updated.

Also, I'd like to replace TBB_USE_SYSTEM_LIB with TBB_LIB and TBB_INC environment variables and use them to update LDFLAGS and CPPFLAGS in the build files. This will allow using any external version of TBB library, not only the system version. But, it'll ignore the internal TBB source code if TBB_LIB and TBB_INC are defined. What do you think?

I personally prefer it to be modular for any external library (i.e., allow the user to use internal or external libraries) unless you make custom changes to its source code. In the case of TBB, there's no need to build the internal source code if the library and its headers are available in the system.

@rok-cesnovar
Copy link
Member

@rok-cesnovar TBB_USE_GLOBAL_CONTROL would be a bit misleading since the old interface has tbb::global_control too. But, we could predefine __TBB_tbb_stddef_H in the compiler flags for the old interface and remove it when the internal TBB source code is updated.

Ok, then lets leave it at the new interface.

Also, I'd like to replace TBB_USE_SYSTEM_LIB with TBB_LIB and TBB_INC environment variables and use them to update LDFLAGS and CPPDLAGS in the build files. This will allow using any external version of TBB library, not only the system version. But, it'll ignore the internal TBB source code if TBB_LIB and TBB_INC are defined. What do you think?

Seems reasonable yes.

Thanks for working on this!

Use TBB_INC & TBB_LIB environment variables and update the compiler flags accordingly.

To use system TBB, both TBB_INC & TBB_LIB environment variables should be defined and the directories of TBB headers and libraries exist.

Signed-off-by: Hamada S. Badr <hamada.s.badr@gmail.com>
@hsbadr hsbadr changed the title Update TBB interface and allow using the system lib Update TBB interface and allow using external lib Dec 13, 2020
hsbadr added a commit to hsbadr/rstan that referenced this pull request Dec 13, 2020
hsbadr added a commit to hsbadr/rstan that referenced this pull request Dec 13, 2020
hsbadr added a commit to hsbadr/rstan that referenced this pull request Dec 14, 2020
@stan-buildbot
Copy link
Contributor


Name Old Result New Result Ratio Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan 3.42 3.44 1.0 -0.39% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan 0.02 0.02 1.0 0.06% faster
eight_schools/eight_schools.stan 0.12 0.11 1.01 1.33% faster
gp_regr/gp_regr.stan 0.15 0.15 0.99 -1.04% slower
irt_2pl/irt_2pl.stan 5.41 5.45 0.99 -0.8% slower
performance.compilation 87.45 85.9 1.02 1.77% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan 8.41 8.38 1.0 0.42% faster
pkpd/one_comp_mm_elim_abs.stan 29.9 32.93 0.91 -10.13% slower
sir/sir.stan 140.79 144.5 0.97 -2.64% slower
gp_regr/gen_gp_data.stan 0.04 0.04 0.98 -1.91% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan 2.91 2.98 0.98 -2.19% slower
pkpd/sim_one_comp_mm_elim_abs.stan 0.41 0.38 1.09 8.63% faster
arK/arK.stan 2.51 2.47 1.02 1.94% faster
arma/arma.stan 0.59 0.59 1.0 -0.12% slower
garch/garch.stan 0.6 0.61 0.99 -1.15% slower
Mean result: 0.997203630035

Jenkins Console Log
Blue Ocean
Commit hash: 0a069c1


Machine information ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Copy link
Member

@rok-cesnovar rok-cesnovar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thank you and congrats on your first merged PR to the Stan Math repository! 🎉

ifeq ($(OS),Windows_NT)
TBB_TARGETS ?= $(addprefix $(TBB_BIN)/,$(addsuffix $(LIBRARY_SUFFIX),$(TBB_LIBRARIES)))
endif
ifeq ($(OS),Darwin)
TBB_TARGETS ?= $(addprefix $(TBB_BIN)/lib,$(addsuffix $(LIBRARY_SUFFIX), $(TBB_LIBRARIES)))
endif
ifeq ($(OS),Linux)
# Update the suffix with the internal TBB source code!
# The new version of TBB library is 12+ (e.g., libtbb.so.12.1 not libtbb.so.2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the heads up!

@rok-cesnovar rok-cesnovar merged commit 7f3792b into stan-dev:develop Dec 14, 2020
hsbadr added a commit to hsbadr/RcppParallel that referenced this pull request Dec 14, 2020
hsbadr added a commit to hsbadr/RcppParallel that referenced this pull request Dec 14, 2020
hsbadr added a commit to hsbadr/RcppParallel that referenced this pull request Dec 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants