task.h is deprecated, alternative?

[tjhei]

With a recent version of TBB I am getting the warning

/usr/include/tbb/task.h:21:139: note: #pragma message: TBB Warning: tbb/task.h is deprecated.
For details, please see Deprecated Features appendix in the TBB reference manual.

but I am unable to find information about the deprecation or an alternative to using the class task. What is the proposed path forward?

[aleksei-fedotov]

The proposed path forward is to use high-level API such as parallel algorithms (e.g. parallel_for, parallel_reduce), flow_graph, task_group, task_arena, etc.
Additional information about the deprecation can be found here.

[alexey-katranov]

@tjhei , can you please describe your use case for the class task to propose better alternatives?

[tjhei]

thanks for the quick reply.

The proposed path forward is to use high-level API such as parallel algorithms (e.g. parallel_for, parallel_reduce), flow_graph, task_group, task_arena, etc.

Okay, we can probably migrate to task_group.

Additional information about the deprecation can be found here.

I went through the document you link before opening the issue, but I can not find any mention of a) task being deprecated or b) what to use instead. Maybe I am missing something?

The example
https://software.intel.com/en-us/node/506102
in the user guide of the documentation also seems to rely on task. Maybe this is worth updating?

[bangerth]

@aleksei-fedotov -- thanks indeed for chiming in! The use case in our software is here: https://github.com/dealii/dealii/blob/master/include/deal.II/base/thread_management.h#L1200-L1231 and https://github.com/dealii/dealii/blob/master/include/deal.II/base/thread_management.h#L1349-L1361

We are, in essence, building there what in C++11 one would achieve with std::packaged_task and std::async(). I believe that the code is idiomatic as described in the documentation of the TBB (and the book by Reinders) around 2010.

[tjhei]

@tjhei , can you please describe your use case for the class task to propose better alternatives?

To extend what @bangerth just wrote:
We use TBB in various ways in deal.II (https://github.com/dealii/dealii) including:

1. tbb::filter (which we need to port to tbb::parallel_pipeline)
2. parallel_for (we are good here)
3. a thin wrapper around tbb:task to run and wait for individual tasks as described by @bangerth . We might be able to move to task_group but I don't see how you can wait for individual tasks to finish in a task_group. I guess we could make a task_group for each task?

[alexey-katranov]

Overall, we do not recommend using task API in TBB directly because it is too low level and hides multiple caveats. tbb::task_group or tbb::parallel_invoke can be used as a straight forward replacement.

@tjheu, as for your use cases

1. Yes, it is better to migrate to tbb::parallel_pipeline
2. no changes
3. Yes, you can use task_group for each task you want to wait.

[Dr15Jones]

It is likely that this change will also cause serious difficulties with the CMS experiment at CERN. We rely on tbb::tasks directly and we've designed the system using tasks in order to avoid the need to do a wait (except one wait which is only waiting on the application to end). We chose tbb 8 years ago specifically because the low level task support gave us the flexibility we needed.

Our initial look at task_group seems to indicate they are not flexible enough for our need.

[alexey-katranov]

We are not going to drop task API but it will be significantly reworked and it will not directly replaceable with the current task API.

Our initial look at task_group seems to indicate they are not flexible enough for our need.

Can you list the interfaces that you are using to understand what we can do to simply the migration?

[bangerth]

@alexey-katranov -- this might be a good time to ask whether under the hood, TBB and C++11 use the same thread pools?

If tbb::task is deprecated, it is not very difficult to replace it using std::packaged_task and std::async(std::launch::async, ...). The interface for the latter is pretty simple. But that won't work if TBB and C++11 facilities use separate thread pools because then one is oversubscribing resources as soon as one mixes std::async and TBB facilities like the pipeline.

[bangerth]

@Dr15Jones -- take a look at tbb::parallel_invoke: https://software.intel.com/en-us/node/506168. I think it does, in essence, what the rather complicated task creation logic did before.

If I read this right (confirmation from @alexey-katranov welcome!), then the only thing it doesn't do is give you a handle to the task itself that is being created. So you can't easily wait for it. But if you wrap the operation you want into a std::packaged_task, then you can call std::packaged_task::get_future() and you can wait for the task to finish by calling std::future::wait().

[alexey-katranov]

TBB and C++11 use the same thread pools?

No, it does not.

Unfortunately, the entity "thread pool" is not defined in C++11. Even Parallel STL (C++17) does not define anything about that. Hopefully, C++ executors will help us to manage/share threads more efficient with future standards.

As for parallel_invoke, it is a blocking API; however, @Dr15Jones is interested in non blocking execution: "except one wait which is only waiting on the application to end".

[bangerth]

On 4/21/20 10:31 AM, Alex wrote: TBB and C++11 use the same thread pools? No, it does not. Unfortunately, the entity "thread pool" is not defined in C++11. Even Parallel STL (C++17) does not define anything about that. Hopefully, C++ executors will help us to manage/share threads more efficient with future standards.

Right -- it was a question about implementation practice, not what the standard guarantees. How does TBB implement tasks? Using the C++11 facilities, or does it use its own thread pool?

As for |parallel_invoke|, it is a blocking API;

Oh, I had assumed that there is a version of |parallel_invoke| with a single argument that just creates a task and immediately returns. But you're saying that all of these functions only return when all tasks have finished. So then what is the equivalent of |std::async(std::launch::async, ...);|?

[alexey-katranov]

As for std::async(std::launch::async, ... ), it has slightly different semantics then TBB tasking because it requires an explicit thread behind it: "std::launch::async | a new thread is launched to execute the task asynchronously" (cppreference). As for runnig a task and not blocking the calling thread, consider tbb::task_group::run.

As for Parallel STL, it should be mentioned that some implementations use TBB underneath, so TBB thread pool is shared in such C++ implementations.

[Dr15Jones]

@alexey-katranov Thanks for your reply

Our initial look at task_group seems to indicate they are not flexible enough for our need.

Can you list the interfaces that you are using to understand what we can do to simply the migration?

It will take some time for me to do a comprehensive inventory of what we are using. Most of our wrappings around TBB are here (https://github.com/cms-sw/cmssw/tree/master/FWCore/Concurrency) though we do have non-wrapped usages as well.

Consulting with my colleagues, we will explore trying to use just tbb::task_group and higher constructs via our 'toy' system (https://github.com/Dr15Jones/toy-mt-framework). The toy doesn't do everything our full system can but it is safer and easier to modify.

Other than direct use of tasks, are there any other TBB constructs we should avoid?

[alexey-katranov]

Most of our wrappings around TBB are here (https://github.com/cms-sw/cmssw/tree/master/FWCore/Concurrency) though we do have non-wrapped usages as well.

I've looked through the code, overall, the usage does not seem complex. However, I've noticed tbb::task::enqueue, can you clarify why you cannot use tbb::task::spawn? Do you want to run a task but you do not want/need to wait for the task completion?

[Dr15Jones]

However, I've noticed tbb::task::enqueue, can you clarify why you cannot use tbb::task::spawn?

We use that in the cases where we are communicating with a thread not under the control of tbb. For example, that could be a callback from CUDA or a thread which is waiting for a response from another process (e.g. evaluating a ML inference being done remotely). In all other cases we use tbb::task::spawn.

[alexey-katranov]

@Dr15Jones , it seems as a some sort of workaround because tbb::task::enqueue goes to the arena associated with the calling thread and it enforces a worker thread to come even if you do not wait in this arena. I suppose you would like to add a task to the arena where the waiting thread works. Do I understand the idea correctly?

[Dr15Jones]

I suppose you would like to add a task to the arena where the waiting thread works. Do I understand the idea correctly?

Yes. We typically have one main arena (the default one) for the majority of our tasks. When we have work that is coming from outside of TBB (e.g. a CUDA callback) we want to add a task to the arena from which the external request was made (e.g. the arena assigned to the thread used to call a CUDA asynchronous function that takes the callback parameter).

We do have a few other arenas (primarily used to fence off third party functions which are also using TBB) so we do not assume there is only one arena active in our application.

[Dr15Jones]

@alexey-katranov I also wanted to say 'thank you' for taking the time to take a look at our code.

[alexey-katranov]

@Dr15Jones, you are welcome. I am closing this thread if there is no more questions. If you still have some issues or questions, feel free to reopen the thread or start a new issue.

[alexey-katranov]

Consider #367 and uxlfoundation/oneAPI-spec#292 that introduce task_handle to cover additional use cases.

[jiapei100]

Yeah, I do need the help as well : heremaps/pptk#58