Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TBB oneapi compatibility #1600

Open
jiapei100 opened this issue Aug 23, 2021 · 2 comments
Open

TBB oneapi compatibility #1600

jiapei100 opened this issue Aug 23, 2021 · 2 comments

Comments

@jiapei100
Copy link

jiapei100 commented Aug 23, 2021 •
edited
Loading

Description of Issue

Looks like USD is NOT compatible with the most recent Intel oneapi 2021.3.0 ?

[ 49%] Building CXX object pxr/base/work/CMakeFiles/work.dir/utils.cpp.o
cd ....../USD/build/pxr/base/work && /usr/local/bin/c++ -DBOOST_PYTHON_NO_PY_SIGNATURES -DGLX_GLXEXT_PROTOTYPES -DGL_GLEXT_PROTOTYPES -DMFB_ALT_PACKAGE_NAME=work -DMFB_PACKAGE_MODULE=Work -DMFB_PACKAGE_NAME=work -DPXR_BUILD_LOCATION=usd -DPXR_OPENVDB_SUPPORT_ENABLED -DPXR_PLUGIN_BUILD_LOCATION=../plugin/usd -DPXR_PTEX_SUPPORT_ENABLED -DPXR_PYTHON_ENABLED=1 -DPXR_PYTHON_MODULES_ENABLED=1 -DWORK_EXPORTS=1 -Dwork_EXPORTS -I....../USD/build/pxr/base/work -I....../USD/pxr/base/work -I....../USD/build/include -isystem /opt/intel/oneapi/tbb/latest/include -isystem /usr/include/python3.8 -Wall -pthread -Wno-deprecated -Wno-deprecated-declarations -Wno-unused-local-typedefs  -O3 -DNDEBUG -fPIC -std=c++14 -MD -MT pxr/base/work/CMakeFiles/work.dir/utils.cpp.o -MF CMakeFiles/work.dir/utils.cpp.o.d -o CMakeFiles/work.dir/utils.cpp.o -c ....../USD/pxr/base/work/utils.cpp
....../USD/pxr/base/work/threadLimits.cpp:65:13: error: ‘task_scheduler_init’ in namespace ‘tbb’ does not name a type
   65 | static tbb::task_scheduler_init *_tbbTaskSchedInit;
      |             ^~~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp: In function ‘unsigned int pxrInternal_v0_21__pxrReserved__::WorkGetPhysicalConcurrencyLimit()’:
....../USD/pxr/base/work/threadLimits.cpp:72:17: error: ‘tbb::task_scheduler_init’ has not been declared
   72 |     return tbb::task_scheduler_init::default_num_threads();
      |                 ^~~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp: In function ‘void pxrInternal_v0_21__pxrReserved__::Work_InitializeThreading()’:
....../USD/pxr/base/work/threadLimits.cpp:128:9: error: ‘_tbbTaskSchedInit’ was not declared in this scope
  128 |         _tbbTaskSchedInit = new tbb::task_scheduler_init(threadLimit);
      |         ^~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp:128:33: error: expected type-specifier
  128 |         _tbbTaskSchedInit = new tbb::task_scheduler_init(threadLimit);
      |                                 ^~~
....../USD/pxr/base/work/threadLimits.cpp:119:14: warning: unused variable ‘threadLimit’ [-Wunused-variable]
  119 |     unsigned threadLimit =
      |              ^~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp: In function ‘void pxrInternal_v0_21__pxrReserved__::WorkSetConcurrencyLimit(unsigned int)’:
....../USD/pxr/base/work/threadLimits.cpp:167:9: error: ‘_tbbTaskSchedInit’ was not declared in this scope
  167 |     if (_tbbTaskSchedInit) {
      |         ^~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp:171:33: error: expected type-specifier
  171 |         _tbbTaskSchedInit = new tbb::task_scheduler_init(threadLimit);
      |                                 ^~~
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:194: pxr/base/work/CMakeFiles/work.dir/threadLimits.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/detachedTask.cpp:26:
....../USD/pxr/base/work/dispatcher.h:136:44: error: expected class-name before ‘{’ token
  136 |     struct _InvokerTask : public tbb::task {
      |                                            ^
....../USD/pxr/base/work/dispatcher.h:143:22: error: ‘task’ in namespace ‘tbb’ does not name a type
  143 |         virtual tbb::task* execute() {
      |                      ^~~~
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:36,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/detachedTask.h:30,
                 from ....../USD/pxr/base/work/detachedTask.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/task.h:26:11: note: ‘tbb::v1::task’ declared here
   26 | namespace task {
      |           ^~~~
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/detachedTask.cpp:26:
....../USD/pxr/base/work/dispatcher.h:171:5: error: ‘detail’ does not name a type
  171 |     detail::d1::task_group_context _context;
      |     ^~~~~~
....../USD/pxr/base/work/dispatcher.h:172:10: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
  172 |     tbb::empty_task* _rootTask;
      |          ^~~~~~~~~~
In file included from ....../USD/pxr/base/work/dispatcher.cpp:26:
....../USD/pxr/base/work/dispatcher.h:136:44: error: expected class-name before ‘{’ token
  136 |     struct _InvokerTask : public tbb::task {
      |                                            ^
....../USD/pxr/base/work/dispatcher.h:143:22: error: ‘task’ in namespace ‘tbb’ does not name a type
  143 |         virtual tbb::task* execute() {
      |                      ^~~~
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/utils.h:31,
                 from ....../USD/pxr/base/work/utils.cpp:26:
....../USD/pxr/base/work/dispatcher.h:136:44: error: expected class-name before ‘{’ token
  136 |     struct _InvokerTask : public tbb::task {
      |                                            ^
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:36,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/dispatcher.h:33,
                 from ....../USD/pxr/base/work/dispatcher.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/task.h:26:11: note: ‘tbb::v1::task’ declared here
   26 | namespace task {
      |           ^~~~
....../USD/pxr/base/work/dispatcher.h:143:22: error: ‘task’ in namespace ‘tbb’ does not name a type
  143 |         virtual tbb::task* execute() {
      |                      ^~~~
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:36,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/detachedTask.h:30,
                 from ....../USD/pxr/base/work/utils.h:31,
                 from ....../USD/pxr/base/work/utils.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/task.h:26:11: note: ‘tbb::v1::task’ declared here
   26 | namespace task {
      |           ^~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Run(Callable&&)’:
....../USD/pxr/base/work/dispatcher.h:102:9: error: ‘_rootTask’ was not declared in this scope
  102 |         _rootTask->spawn(_MakeInvokerTask(std::forward<Callable>(c)));
      |         ^~~~~~~~~
In file included from ....../USD/pxr/base/work/dispatcher.cpp:26:
....../USD/pxr/base/work/dispatcher.h:171:5: error: ‘detail’ does not name a type
  171 |     detail::d1::task_group_context _context;
      |     ^~~~~~
....../USD/pxr/base/work/dispatcher.h:172:10: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
  172 |     tbb::empty_task* _rootTask;
      |          ^~~~~~~~~~
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/utils.h:31,
                 from ....../USD/pxr/base/work/utils.cpp:26:
....../USD/pxr/base/work/dispatcher.h:171:5: error: ‘detail’ does not name a type
  171 |     detail::d1::task_group_context _context;
      |     ^~~~~~
....../USD/pxr/base/work/dispatcher.h:172:10: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
  172 |     tbb::empty_task* _rootTask;
      |          ^~~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_InvokerTask<typename std::remove_reference<_Tp>::type>& pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_MakeInvokerTask(Fn&&)’:
....../USD/pxr/base/work/dispatcher.h:159:22: error: ‘_rootTask’ was not declared in this scope
  159 |         return *new( _rootTask->allocate_additional_child_of(*_rootTask) )
      |                      ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Run(Callable&&)’:
....../USD/pxr/base/work/dispatcher.h:102:9: error: ‘_rootTask’ was not declared in this scope
  102 |         _rootTask->spawn(_MakeInvokerTask(std::forward<Callable>(c)));
      |         ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Run(Callable&&)’:
....../USD/pxr/base/work/dispatcher.h:102:9: error: ‘_rootTask’ was not declared in this scope
  102 |         _rootTask->spawn(_MakeInvokerTask(std::forward<Callable>(c)));
      |         ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_InvokerTask<typename std::remove_reference<_Tp>::type>& pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_MakeInvokerTask(Fn&&)’:
....../USD/pxr/base/work/dispatcher.h:159:22: error: ‘_rootTask’ was not declared in this scope
  159 |         return *new( _rootTask->allocate_additional_child_of(*_rootTask) )
      |                      ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_InvokerTask<typename std::remove_reference<_Tp>::type>& pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_MakeInvokerTask(Fn&&)’:
....../USD/pxr/base/work/dispatcher.h:159:22: error: ‘_rootTask’ was not declared in this scope
  159 |         return *new( _rootTask->allocate_additional_child_of(*_rootTask) )
      |                      ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In constructor ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::WorkDispatcher()’:
....../USD/pxr/base/work/dispatcher.cpp:31:7: error: class ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher’ does not have any field named ‘_context’
   31 |     : _context(
      |       ^~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:32:14: error: ‘tbb::task_group_context’ has not been declared
   32 |         tbb::task_group_context::isolated,
      |              ^~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:33:14: error: ‘tbb::task_group_context’ has not been declared
   33 |         tbb::task_group_context::concurrent_wait |
      |              ^~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:34:14: error: ‘tbb::task_group_context’ has not been declared
   34 |         tbb::task_group_context::default_traits)
      |              ^~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:5: error: ‘_rootTask’ was not declared in this scope
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |     ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:32: error: ‘allocate_root’ is not a member of ‘tbb::v1::task’
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |                                ^~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:46: error: ‘_context’ was not declared in this scope
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |                                              ^~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:62: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |                                                              ^~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In destructor ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::~WorkDispatcher()’:
....../USD/pxr/base/work/dispatcher.cpp:46:16: error: ‘destroy’ is not a member of ‘tbb::v1::task’; did you mean ‘tbb::detail::d1::destroy’?
   46 |     tbb::task::destroy(*_rootTask);
      |                ^~~~~~~
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:29,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/dispatcher.h:33,
                 from ....../USD/pxr/base/work/dispatcher.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/profiling.h:68:63: note: ‘tbb::detail::d1::destroy’ declared here
   68 |     enum notify_type {prepare=0, cancel, acquired, releasing, destroy};
      |                                                               ^~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:46:25: error: ‘_rootTask’ was not declared in this scope
   46 |     tbb::task::destroy(*_rootTask);
      |                         ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Wait()’:
....../USD/pxr/base/work/dispatcher.cpp:52:5: error: ‘_rootTask’ was not declared in this scope
   52 |     _rootTask->wait_for_all();
      |     ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:54:9: error: ‘_context’ was not declared in this scope
   54 |     if (_context.is_group_execution_cancelled()) {
      |         ^~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Cancel()’:
....../USD/pxr/base/work/dispatcher.cpp:68:5: error: ‘_context’ was not declared in this scope
   68 |     _context.cancel_group_execution();
      |     ^~~~~~~~
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:208: pxr/base/work/CMakeFiles/work.dir/utils.cpp.o] Error 1
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:124: pxr/base/work/CMakeFiles/work.dir/detachedTask.cpp.o] Error 1
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:138: pxr/base/work/CMakeFiles/work.dir/dispatcher.cpp.o] Error 1
make[2]: Leaving directory '....../USD/build'
make[1]: *** [CMakeFiles/Makefile2:3458: pxr/base/work/CMakeFiles/work.dir/all] Error 2
make[1]: Leaving directory '....../USD/build'
make: *** [Makefile:149: all] Error 2

Steps to Reproduce

  1. Just build USD directly

System Information (OS, Hardware)

Ubuntu 20.04
GPU Geforce 2080 Ti
Python 3.8.10

Package Versions

➜  build git:(release) ✗ pip show pyside2
Name: PySide2
Version: 5.15.2
Summary: Python bindings for the Qt cross-platform application and UI framework
Home-page: https://www.pyside.org
Author: Qt for Python Team
Author-email: pyside@qt-project.org
License: LGPL
Location: ~/.local/lib/python3.8/site-packages
Requires: shiboken2
Required-by:

Build Flags
@spiffmon
Copy link
Member

That is correct, @jiapei100 . OneAPI is a major change for TBB, and it does not make sense for us to invest in moving to it until the entire VFX industry is, because we must all share the same TBB instance when USD is hosted in an application. That will be guided by the VFX reference platform, which does not yet have OneAPI on its roadmap.

@jilliene
Copy link

Filed as internal issue #USD-6857

ras0219-msft added a commit to ras0219-msft/vcpkg that referenced this issue Sep 15, 2021
@ras0219-msft
[usd] Disable USD in CI due to policy PixarAnimationStudios/OpenUSD#1600
ac412fa
@ras0219-msft ras0219-msft mentioned this issue Sep 9, 2022
Merged
BillyONeal added a commit to microsoft/vcpkg that referenced this issue Sep 23, 2022
@BillyONeal @ras0219-msft
@JonLiu1993 @mathisloge @vicroms @vladimir-aubrecht @Thomas1664 @Neumann-A
[tbb, pagmo2] Update TBB to 2021.5 and update pagmo2 to 2.18.0 (#26284)
5f6dfcb 
* [tbb] Update to 2021.3.0

* [tbb] Add Threads dependency

* [pagmo2] Update to support new TBB

* [openvdb] Update to 8.1.0

* [embree2] Remove from baseline -- it is no longer receiving support from upstream

* [usd] Mark as unsupported

* [usd] Disable USD in CI due to policy PixarAnimationStudios/OpenUSD#1600

* [openvdb] Bump port-version

* Update version database

* [pagmo2] Fix vcpkg.json

* update version

* wip update

* versions

* [embree2] deprecate, [openvdb,usd] resolve conflicts, [tbb] update

* Added libxml port.

* Remove port version as it's initial port.

* Added baseline version

* Support only for windows and static

* Allowed building debug version

* update versions

* Update ports/libxpm/portfile.cmake

Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com>

* Added new lines

* Update ports/libxpm/vcpkg.json

Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com>

* Update ports/libxpm/portfile.cmake

Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com>

* Update ports/libxpm/vcpkg.json

Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com>

* Regenerated versions

* Update ports/libxpm/vcpkg.json

Co-authored-by: JonLiu1993 <63675417+JonLiu1993@users.noreply.github.com>

* Libxpm is taken from gitlab now.

* Dropped changes against master in original repo for libxpm.

* Dropped libxpm from baseline

* Dropped version for libxpm

* Update cpuinfo

* Updated date of version

* Update version database

* Removed support for arm32 & uwp as library is not supporting it.

* Version regenerated

* Update ports/cpuinfo/vcpkg.json

Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

* Updated versions

* Initial commit WIP

* Fixed compilation of latest stable version

* Dropped comments

* Switch from version-string to version

* Added versions

* Fixed path of cmake files for other projects

* Updated versions

* Various modernization.

* Fully mark embree2 as deprecated.

* [pagmo2] Disable -Werror

* [cctag] Fix finding and use of TBB.

* [openvdb] Baseline the static versions because they exceed FFFFFFFF in size, and I'm not sure if that's fixable....

* [gazebo] Apply gazebosim/gazebo-classic#3174 patch to fix TBB.

* Respond to failures in https://dev.azure.com/vcpkg/public/_build/results?buildId=76586

* [pagmo2] Add license. See https://github.com/esa/pagmo2/blob/19d774fbb6128124305225803f8c1ad9e2f95c8c/src/bfe.cpp#L5-L27

* [tbb] Add license.

* [embree2] Add "license".

* [tbb] Use vcpkg_cmake_config_fixup and fix version as requested by @LilyWangLL

* [usd] Add usd is known broken message.

* [embree2] Remove completely.

* Fix version database.

* Fix usd version database.

* Fix wrong case on Linux.

* Lowercase the tbb directory to get to their configs.

* [cctag] minimize patches

* [tbb] Apply supports expression fix suggested in #26284 (comment)

* [usd] Add note about upstream issue.

* [pagmo2] Minimize patch.

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Jonliu1993 <13720414433@163.com>
Co-authored-by: Mathis Logemann <mathisloge@gmail.com>
Co-authored-by: Victor Romero <viromer@microsoft.com>
Co-authored-by: Vladimír Aubrecht <vladimir.aubrecht@me.com>
Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com>
Co-authored-by: JonLiu1993 <63675417+JonLiu1993@users.noreply.github.com>
Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>
@Danielmelody Danielmelody mentioned this issue Oct 1, 2022
Merged
@Cheney-W Cheney-W mentioned this issue Dec 7, 2022
Closed
@Cheney-W Cheney-W mentioned this issue Feb 8, 2023
Closed
This was referenced Mar 2, 2023
Merged
Merged
@wangcan26 wangcan26 mentioned this issue Nov 23, 2023
Open
7 tasks
@jwnimmer-tri jwnimmer-tri mentioned this issue Feb 10, 2024
Open
22 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jiapei100 @spiffmon @jilliene