openmp with multiprocessing and benchmarking

[jonwright]

When running peaksearch or grid_index_parallel or refine_em there are some multiprocessing options that do not play well with threading. Multiprocessing launches Ncores jobs and each job launches Ncores threads so you end up with (Ncores x Ncores) fighting for resources. OK when there are 2 cores giving 4 tasks, a bit of an overhead with 64*64 = 4096.

Options/issues:

• Offer and test a cImageD11.set_num_threads() -> call omp_set_num_threads
• (better but py3 only?) https://github.com/joblib/threadpoolctl , also does numpy
• Use : dask? joblib? etc replacing multiprocessing to also spread work on a cluster
• OAR/SLURM: What is omp_get_max_threads and multiprocessing.cpu_count() versus cores assigned?

Related: add some benchmarking / testing code to verify whether cImageD11 functions are working properly and how they are performing versus thread count.

[jonwright]

Done for the omp number of threads. Remains to be seen for multiprocessing.