GraspFilter::filterGraspsHelper - IK-solver fails when using too many OpenMP threads

[Levaru]

I have a very strange problem:
Grasp poses are filtered in GraspFilter::filterGraspsHelper with the help of a multi-threaded OpenMP for-loop (line in question). Part of this filtering function is the IK-solver (line in question).

The thread count is set with omp_set_num_threads(num_threads); and num_threads is determined by std::size_t num_threads = omp_get_max_threads(); (line in question).

I have a Threadripper with 32 cores and omp_get_max_threads(); returns 64 threads. When filtering grasp poses with that thread count, 98-100% of them will fail due to (probably) to IK-timeout even though most of them are completely valid. But when I set the thread count to 2-5 all of the previously failed grasps will be now valid this time around. The higher I set the thread count, the more grasps will fail. Changing the IK-solver timeout from 0.005 to 5 seconds does nothing.

What could be reason for this behavior? Do the threads starve each other for CPU resources when too many are present? Is there something weird going on if too many IK-solvers are working? This is a huge roadblock for me and I don't even know how to properly debug it. Does anyone have a deep understanding of OpenMP and the IK-solvers and knows what the issue here is?

[Au3f]

I have an maybe related issue:
When sampling larger areas with the moveit graps pipeline, the number of grasp-candidates varies a lot when multi-threading is enabled. Without multi-threading, results are consistent. I was not able to do further investigations since the workaround was obvious.

This might be relevant: the problem is occurring on a i7-11800H running Ubuntu 20.04.4 LTS