Segmentation fault occurred when running with PAPI

[icyclv]

Hi, thanks to the developers of this tool, I think it will be of great help to my project. I've encountered a problem when using the omp example, a segmentation fault occurs.

./run_ldpreload.sh
Extrae: WARNING! omp_get_thread_num_real is a NULL pointer. Did the initialization of this module trigger? Retrying initialization...
Welcome to Extrae 4.1.7
Extrae: Detected GOMP version is 4.5
Extrae: Detected and hooked OpenMP runtime: [GNU GOMP]
Extrae: OMP_NUM_THREADS set to 4
Extrae: Parsing the configuration file (extrae.xml) begins
Extrae: Tracing package is located on @sub_PREFIXDIR@
Extrae: Generating intermediate files for Paraver traces.
Extrae: OpenMP routines will collect HW counters information.
Extrae: Warning! change-at-time time units not specified. Using seconds
Extrae: PAPI domain set to ALL for HWC set 1
Extrae: HWC set 1 contains following counters < PAPI_TOT_INS (0x80000032) PAPI_TOT_CYC (0x8000003b) PAPI_L1_DCM (0x80000000) > - never changes
Extrae: Resource usage is disabled at flush buffer.
Extrae: Memory usage is disabled at flush buffer.
Extrae: Tracing buffer can hold 5000000 events
Extrae: Circular buffer disabled.
Extrae: Sampling buffers will be written at instrumentation points
Extrae: Dynamic memory instrumentation is disabled.
Extrae: Basic I/O memory instrumentation is disabled.
Extrae: System calls instrumentation is disabled.
Extrae: Parsing the configuration file (extrae.xml) has ended
Extrae: Intermediate traces will be stored in /mnt/hdd/users/ycchang/code/performance/extrae-4.1.7/example/LINUX/OMP
Extrae: Tracing mode is set to: Detail.
Extrae: Successfully initiated with 1 tasks and 4 threads

./run_ldpreload.sh: line 9: 2000956 Segmentation fault      (core dumped) ./pi

I've looked through previous issues and found similar problems #13 . It seems that if I disable PAPI, then this fault does not occur. Also, according to the core dump information, it seems to indicate a problem with Papi.

#0  0x00007f85812fafe5 in check_scheduability (ctl=ctl@entry=0x7f8464000d80, idx=idx@entry=0, ctx=0x7f8464000c20) at components/perf_event/perf_event.c:526
#1  0x00007f85812fbb8b in open_pe_events (ctl=0x7f8464000d80, ctx=<optimized out>) at components/perf_event/perf_event.c:811
#2  _pe_update_control_state (ctx=<optimized out>, count=<optimized out>, native=<optimized out>, ctl=<optimized out>) at components/perf_event/perf_event.c:1657
#3  _pe_update_control_state (ctl=0x7f8464000d80, native=<optimized out>, count=<optimized out>, ctx=<optimized out>) at components/perf_event/perf_event.c:1540
#4  0x00007f85812e7c95 in add_native_events (ESI=ESI@entry=0x7f8464000c80, nevt=nevt@entry=0x7f8581338d60 <_papi_hwi_presets+10048>, size=1, out=0x7f84640151e0) at papi_internal.c:1297
#5  0x00007f85812e9b7f in _papi_hwi_add_event (ESI=0x7f8464000c80, EventCode=<optimized out>, EventCode@entry=-2147483598) at papi_internal.c:1415
#6  0x00007f85812e249d in PAPI_add_event (EventSet=<optimized out>, EventCode=-2147483598) at papi.c:2243
#7  0x00007f858162cf4a in HWCBE_PAPI_Init_Thread (time=15605850144237681, threadid=threadid@entry=1, forked=forked@entry=0) at papi_hwc.c:612
#8  0x00007f858162ad8c in HWC_Read (tid=tid@entry=1, time=<optimized out>, store_buffer=store_buffer@entry=0x7f8469668ac8) at common_hwc.c:574
#9  0x00007f858163db05 in Probe_OpenMP_UF_Entry (uf=uf@entry=94372245431352) at omp-probe.c:88
#10 0x00007f8581642ee7 in Extrae_OpenMP_UF_Entry (uf=0x55d4c02a6438 <do_work._omp_fn.0>) at omp-events.c:63
#11 0x00007f858164386a in callme_par (par_helper_ptr=0x7ffc62ccf340) at gnu-libgomp.c:448
#12 0x00007f858158dc0e in ?? () from /lib/x86_64-linux-gnu/libgomp.so.1
#13 0x00007f85813dbac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#14 0x00007f858146d850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Could you advise on how I might resolve this issue to allow me to collect some performance data through PAPI?

Thanks,
Yachen

[icyclv]

When omp thread number is set to 1, run_ldpreload.sh can run normally

[gllort]

We've just released version 4.2.1 of the tool. Since 4.1.7, there have been several critical bug fixes. I suggest trying the new version first to see if the problem has already been resolved.

If not, here are two more things to check:

1. Which compiler did you use to build Extrae? We've detected some issues when using recent versions of the Intel compiler. If this is the case, can you try building Extrae with GNU compilers?

2. For OpenMP codes, we're also seeing in some systems that it is necessary to increase the thread stack size using:

export OMP_STACKSIZE=2G

Do any of these tips improve the situation?

[icyclv]

Thank you! By adjusting OMP_STACKSIZE, this example works now!

Additionally, I hadn't tried Intel's compiler before, but when I just tried it, I encountered an error message:

Unable to set OMP thread stack size to 4194432 bytes

If I had tried using icc before and encountered this error, I probably wouldn't have raised this issue, haha.

Thanks again for your help!