OpenMP 5 fixes #25

rtobar · 2020-10-01T10:06:59Z

These are fixes to suppress warnings due to VR using omp_{get,set}_nested, which is deprecated in OpenMP 5. The fixes are mostly in NBodylib, which checks the compile-time OpenMP version (there's no way to check the runtime version) and decides whether to use the old functions, or the newer alternatives.

I took the chance of cleaning a bit the code around these function calls, trying to reduce duplication and mental load.

icrar-velociraptor-bot · 2020-10-01T10:22:41Z

3d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	5

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

6d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	3
Halo_6D_linking_length_factor	1.0
Halo_6D_vel_linking_length_factor	1.25

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

rtobar · 2020-10-01T10:51:50Z

@MatthieuSchaller I don't have an environment with compile-time OpenMP 5.0 at hand, so if you could give these a try would be great, thanks!

MatthieuSchaller · 2020-10-01T12:55:15Z

I am now on c7ea1c73e67e7f7fdfce0f035bc06dda50ca6672 for NBodyLib and b5e1a8c for VR. The messages still appear.

rtobar · 2020-10-01T13:09:43Z

@MatthieuSchaller the NBodylib commit seems the correct one. Could you run the stf executable and see what OpenMP version it declares it was compiled against? I just pushed a small change so you can see this information when compiling VR with the swift integration; before the executable would exit quickly before printing anything useful.

For reference, I am using the new omp_{get,set}_max_active_levels when _OPENMP >= 201811, which should be OpenMP 5.0; otherwise using the old omp_{get,set}_nested.

rtobar · 2020-10-01T13:11:21Z

Actually you can also simply set the OMP_DISPLAY_ENV=VERBOSE environment variable and that should give you all the required OpenMP information.

rtobar · 2020-10-01T13:16:31Z

In my system, this gives me:

[rtobar@bolano NBodylib]---> OMP_DISPLAY_ENV=VERBOSE ../builds/clang/stf 
Running VELOCIraptor 43de8e8a490a857fe31f84efb396d023143a7e4f-DIRTY

OPENMP DISPLAY ENVIRONMENT BEGIN
   _OPENMP='201611'
  [host] KMP_ABORT_DELAY='0'
...
...
VELOCIraptor/STF running with MPI. Number of mpi threads: 1
VELOCIraptor/STF running with OpenMP. Number of openmp threads: 4
VELOCIraptor/STF running with OpenMP version 201511

Note the difference between the compile-time version (201511) and the runtime version (201611). This difference is known to cause confusion, so we're not alone.

icrar-velociraptor-bot · 2020-10-01T13:24:18Z

3d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	5

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

6d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	3
Halo_6D_linking_length_factor	1.0
Halo_6D_vel_linking_length_factor	1.25

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

MatthieuSchaller · 2020-10-01T13:29:40Z

Quite verbose, but kept everything in case it helps:

[login7c] build > OMP_DISPLAY_ENV=VERBOSE ./stf 

OPENMP DISPLAY ENVIRONMENT BEGIN
   _OPENMP='201611'
  [host] KMP_ABORT_DELAY='0'
  [host] KMP_ADAPTIVE_LOCK_PROPS='1,1024'
  [host] KMP_ALIGN_ALLOC='64'
  [host] KMP_ALL_THREADPRIVATE='128'
  [host] KMP_ATOMIC_MODE='2'
  [host] KMP_BLOCKTIME='200'
  [host] KMP_CPUINFO_FILE: value is not defined
  [host] KMP_DETERMINISTIC_REDUCTION='FALSE'
  [host] KMP_DEVICE_THREAD_LIMIT='2147483647'
  [host] KMP_DISP_HAND_THREAD='FALSE'
  [host] KMP_DISP_NUM_BUFFERS='7'
  [host] KMP_DUPLICATE_LIB_OK='FALSE'
  [host] KMP_ENABLE_TASK_THROTTLING='TRUE'
  [host] KMP_FORCE_MONOTONIC_DYNAMIC_SCHEDULE='FALSE'
  [host] KMP_FORCE_REDUCTION: value is not defined
  [host] KMP_FOREIGN_THREADS_THREADPRIVATE='TRUE'
  [host] KMP_FORKJOIN_BARRIER='2,2'
  [host] KMP_FORKJOIN_BARRIER_PATTERN='hyper,hyper'
  [host] KMP_FORKJOIN_FRAMES='TRUE'
  [host] KMP_FORKJOIN_FRAMES_MODE='3'
  [host] KMP_GTID_MODE='3'
  [host] KMP_HANDLE_SIGNALS='FALSE'
  [host] KMP_HOT_TEAMS_MAX_LEVEL='1'
  [host] KMP_HOT_TEAMS_MODE='0'
  [host] KMP_INIT_AT_FORK='TRUE'
  [host] KMP_ITT_PREPARE_DELAY='0'
  [host] KMP_LIBRARY='throughput'
  [host] KMP_LOCK_KIND='tas'
  [host] KMP_MALLOC_POOL_INCR='1M'
  [host] KMP_MWAIT_HINTS='0'
  [host] KMP_NUM_LOCKS_IN_BLOCK='1'
  [host] KMP_PLAIN_BARRIER='2,2'
  [host] KMP_PLAIN_BARRIER_PATTERN='hyper,hyper'
  [host] KMP_REDUCTION_BARRIER='1,1'
  [host] KMP_REDUCTION_BARRIER_PATTERN='hyper,hyper'
  [host] KMP_SCHEDULE='static,balanced;guided,iterative'
  [host] KMP_SETTINGS='FALSE'
  [host] KMP_SPIN_BACKOFF_PARAMS='4096,100'
  [host] KMP_STACKOFFSET='64'
  [host] KMP_STACKPAD='0'
  [host] KMP_STACKSIZE='8M'
  [host] KMP_STORAGE_MAP='FALSE'
  [host] KMP_TASKING='2'
  [host] KMP_TASKLOOP_MIN_TASKS='0'
  [host] KMP_TASK_STEALING_CONSTRAINT='1'
  [host] KMP_TEAMS_THREAD_LIMIT='28'
  [host] KMP_TOPOLOGY_METHOD='all'
  [host] KMP_USER_LEVEL_MWAIT='FALSE'
  [host] KMP_USE_YIELD='1'
  [host] KMP_VERSION='FALSE'
  [host] KMP_WARNINGS='TRUE'
  [host] OMP_AFFINITY_FORMAT='OMP: pid %P tid %i thread %n bound to OS proc set {%A}'
  [host] OMP_ALLOCATOR='omp_default_mem_alloc'
  [host] OMP_CANCELLATION='FALSE'
  [host] OMP_DEBUG='disabled'
  [host] OMP_DEFAULT_DEVICE='0'
  [host] OMP_DISPLAY_AFFINITY='FALSE'
  [host] OMP_DISPLAY_ENV='VERBOSE'
  [host] OMP_DYNAMIC='FALSE'
  [host] OMP_MAX_ACTIVE_LEVELS='1'
  [host] OMP_MAX_TASK_PRIORITY='0'
  [host] OMP_NESTED: deprecated; max-active-levels-var=1
  [host] OMP_NUM_TEAMS='0'
  [host] OMP_NUM_THREADS: value is not defined
  [host] OMP_PLACES: value is not defined
  [host] OMP_PROC_BIND='false'
  [host] OMP_SCHEDULE='static'
  [host] OMP_STACKSIZE='8M'
  [host] OMP_TARGET_OFFLOAD=DEFAULT
  [host] OMP_TEAMS_THREAD_LIMIT='0'
  [host] OMP_THREAD_LIMIT='2147483647'
  [host] OMP_TOOL='enabled'
  [host] OMP_TOOL_LIBRARIES: value is not defined
  [host] OMP_WAIT_POLICY='PASSIVE'
  [host] KMP_AFFINITY='noverbose,warnings,respect,granularity=core,none'
OPENMP DISPLAY ENVIRONMENT END


VELOCIraptor/STF running with OpenMP. Number of openmp threads: 28
VELOCIraptor/STF running with OpenMP version 201611

rtobar · 2020-10-02T03:47:09Z

I just checked the OpenMP version that corresponds to 201611 and that seems to be 5.0 rev1. However there's nothing there about omp_get_nested being deprecated. The routines appear as deprecated in 5.0 rev2 from Nov 2017 though, so 201711 is the smallest _OPENMP macro version one can check against I think (I'm using 201811 at the moment).

On the one hand, the fact that Intel's compiler and OpenMP library report 201611 but implement the deprecation doesn't align with what the standard says, so one could say they are at fault. But on the other I guess we could probably just always (OpenMP >= 3.0) use omp_{get,set}_max_active_levels. I'll try this and let you know.

This information might be useful in the future. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

rtobar · 2020-10-02T04:03:19Z

@MatthieuSchaller I just pushed the latest changes to require only OpenMP 3.0 to choose the new functions, could you give them a try?

These fixes are meant to suppress deprecation warnings stemming from the use of omp_set/get_nested reported in #22. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

When the stf executable is run, it immediately exists if the software was compiled to support integration with swift. This fast failure however prevents the executable from showing some important information that can be useful for debugging problems like #23. Moving the fast exit only a couple of lines down allows for these informational messages to be printed before the program exits. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

icrar-velociraptor-bot · 2020-10-02T04:20:30Z

3d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	5

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

6d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	3
Halo_6D_linking_length_factor	1.0
Halo_6D_vel_linking_length_factor	1.25

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

icrar-velociraptor-bot · 2020-10-02T04:35:39Z

3d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	5

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

6d-run

Relevant configuration parameters full file:

Parameter	Value
Bound_halos	0
Physical_linking_length	0.10
FoF_Field_search_type	3
Halo_6D_linking_length_factor	1.0
Halo_6D_vel_linking_length_factor	1.25

R_200crit v/s Mass_200crit:

R_200mean v/s Mass_200mean:

Mass_200crit histogram:

Mass_200mean histogram:

R_200crit histogram:

R_200mean histogram:

MatthieuSchaller · 2020-10-02T12:11:13Z

No more messages in this latest version.

rtobar mentioned this pull request Oct 1, 2020

OMP warning at runtime when compiling with Intel MPI 2020.2 #22

Closed

Output compile-time OpenMP version at startup

2b0013e

This information might be useful in the future. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

rtobar force-pushed the omp-5-fixes branch from 7d5b4e5 to 12ded8a Compare October 2, 2020 04:00

rtobar added 2 commits October 2, 2020 12:13

Use OpenMP 5 fixes in NBodylib

4c2f2c8

These fixes are meant to suppress deprecation warnings stemming from the use of omp_set/get_nested reported in #22. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

rtobar force-pushed the omp-5-fixes branch from 12ded8a to 34d55cb Compare October 2, 2020 04:14

rtobar merged commit 34d55cb into master Oct 4, 2020

rtobar deleted the omp-5-fixes branch October 4, 2020 00:28

rtobar mentioned this pull request Oct 7, 2020

Omp 5 fixes #31

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenMP 5 fixes #25

OpenMP 5 fixes #25

rtobar commented Oct 1, 2020

icrar-velociraptor-bot commented Oct 1, 2020

rtobar commented Oct 1, 2020

MatthieuSchaller commented Oct 1, 2020

rtobar commented Oct 1, 2020

rtobar commented Oct 1, 2020

rtobar commented Oct 1, 2020

icrar-velociraptor-bot commented Oct 1, 2020

MatthieuSchaller commented Oct 1, 2020

rtobar commented Oct 2, 2020 •

edited

Loading

rtobar commented Oct 2, 2020

icrar-velociraptor-bot commented Oct 2, 2020

icrar-velociraptor-bot commented Oct 2, 2020

MatthieuSchaller commented Oct 2, 2020

OpenMP 5 fixes #25

OpenMP 5 fixes #25

Conversation

rtobar commented Oct 1, 2020

icrar-velociraptor-bot commented Oct 1, 2020

3d-run

6d-run

rtobar commented Oct 1, 2020

MatthieuSchaller commented Oct 1, 2020

rtobar commented Oct 1, 2020

rtobar commented Oct 1, 2020

rtobar commented Oct 1, 2020

icrar-velociraptor-bot commented Oct 1, 2020

3d-run

6d-run

MatthieuSchaller commented Oct 1, 2020

rtobar commented Oct 2, 2020 • edited Loading

rtobar commented Oct 2, 2020

icrar-velociraptor-bot commented Oct 2, 2020

3d-run

6d-run

icrar-velociraptor-bot commented Oct 2, 2020

3d-run

6d-run

MatthieuSchaller commented Oct 2, 2020

rtobar commented Oct 2, 2020 •

edited

Loading