make kernel execution thread safe

[BenjaminW3]

Is it allowed for the host code to be multithreaded itself?
Restricting it is neither useful nor realistically enforceable.
Maybe one thread calculates something using a CUDA device while the other thread uses OpenMP?
This use case requires locking and the removal of some hidden state.

• None of the KernelTasks are thread safe. If multiple threads share the same KernelTask and issue its execution (possibly with different arguments on different devices of the same accelerator type), they interfere and override their values.
  -> Each operator() of a KernelTask has to be secured by a lock.
• If the host code is multithreaded and issues two kernels to the same CUDA device in parallel, the argument setup functions and kernel launch functions can overlap.
  -> A process global mutex lock is required.
• OpenMP usage nested in other threading primitives (std::thread, pthreads) is undefined behaviour:
  http://stackoverflow.com/questions/13197510/why-do-c11-threads-become-unjoinable-when-using-nested-openmp-pragmas
  http://stackoverflow.com/questions/13837696/can-i-safely-use-openmp-with-c11
  -> Therefore a process global mutex for the execution of OpenMP kernels is required

[ax3l]

puh, that can generally be useful but as you state it correctly it brings a good amount of requirements and additional checks (locks) for the code that harm performance.

the devices one should have in mind are SoC systems like jetson's K1 and general many-core systems in numa style, e.g. one might want to run a self-contained set of OpenMP threads per MP ("sockel") of a XeonPhi (or vice versa: if I have N MPs with M vector instruction units each: I might want to run 1 OpenMP thread per MP and M OpenMP thread groups with size of N threads each).

[fwyzard]

@BenjaminW3 @ax3l is this still an open issue ?

That is, if I understand correctly, it is not possible to launch multiple kernels on the same device from different threads ?

[BenjaminW3]

@fwyzard I can not say for sure because it is not tested but with some restrictions it should be possible to launch multiple tasks on the same device from different threads.

It should be possible to have one queue per thread and enqueue tasks into those queues in parallel.
However, it should also be possible to share a queue across multiple threads and enqueue tasks into this queue in parallel because the queues are designed to be thread-safe.
If something does not work, please report it and we will try to fix it.
There are some natural limitations which we can not changed. For example, CUDA error handling is by design not thread-safe and there is nothing we can do about this. When you get an CUDA error reported from alpaka you can not be sure which kernel or operation produced this error.

[j-stephan]

Is it allowed for the host code to be multithreaded itself?

The SYCL back-end currently assumes this and I think it would be useful to have this guaranteed for all of alpaka. Implementing this for SYCL was relatively straight forward so I believe this could (and should) be achieved as part of the next alpaka release with not too much effort.