Skip to content

Notes on experiments

Jump to bottom
Chunhua "Leo" Liao edited this page Feb 13, 2020 · 2 revisions

We use the following steps for inspector and get all the output from stdout and stderr:

Step 1: compile with ICC

Icc -O0 -g -fopenmp input.c -o exec.out

Step 2: run the application through inspector

inspxe-cl -collect ti3 -knob scope=extreme -knob stack-depth=16 -knob use-maximum-resources=true – exec.out

Intel will report keywords like ‘Data race’ in the output if race is found. And I notice gcc is used to compile the program. This is allowed by inspector but not recommended. It is better to use intel compiler and intel OpenMP runtime.

ROMP has a more complex process to run its tool, here is the steps we had for DataRaceBench:

Step 1: compile with Clang/Clang++ to generate a executable

clang -g -O0 -L${ROMP_ROOT}/pkgs-src/romp-lib/lib -L${ROMP_ROOT}/pkgs-src/gperftools/gperftools-install/lib -L${ROMP_ROOT}/pkgs-src/llvm-openmp/openmp/llvm-openmp-install/lib -fopenmp -fpermissive -ltcmalloc sourcecode.c -o exec.out

Step 2: code instrumentation using Dynainst (provided in ROMP’s instruction) and a output name instrumented_app is generated omp_race_client exec.out

Step 3: run the instrumented code

./ instrumented_app

ROMP will report keyword “race found!” in its output.

I just re-run the test again with the following command:

[lin32@quartz188:dataracebench]$ clang -g -O0 -L/g/g92/lin32/opt/romp/pkgs-src/romp-lib/romp-install/lib/ -L/g/g92/lin32/opt/romp/pkgs-src/gperftools/gperftools-install/lib -L/g/g92/lin32/opt/romp/pkgs-src/llvm-openmp/openmp/llvm-openmp-install/lib -fopenmp -fpermissive -ltcmalloc micro-benchmarks/DRB001-antidep1-orig-yes.c -o DRB001.out -lm

[lin32@quartz188:dataracebench]$ /g/g92/lin32/opt/romp/pkgs-src/dyninst-client/omp_race_client DRB001.out

romp_path: /g/g92/lin32/opt/romp/pkgs-src/romp-lib/romp-install/lib/libomptrace.so
objs num: 11
found main program object: DRB001.out
found .rodata!
.rodata offset: 0x400a50.rodata upper bound: 0x400af5
Number of modules: 2119
module 0 name: DRB001.out
not skipped: DRB001.out
Procedure: _init
Forbidden: _init
Procedure: targ400620
Procedure: _start
….

Many instrumentation output generated here

[lin32@quartz188:dataracebench]$ ./instrumented_app
[0]: process elf: found section
[0]: process elf: section: .stringtable address: 0xf36000 size: 104
[0]: process elf: found section
[0]: process elf: section: .linemap address: 0xf38000 size: 704
0: NULL_POINTER=(nil)
a[500]=502
[0]: ompt finalizer: called
[0]: ompt finalizer: race found!

The libraries linked for original and instrumented executables are in the following:
[lin32@quartz188:dataracebench]$ ldd DRB001.out
                linux-vdso.so.1 =>  (0x00002aaaaaacd000)
                libtcmalloc.so.4 => /g/g92/lin32/opt/romp/pkgs-src/gperftools/gperftools-install/lib/libtcmalloc.so.4 (0x00002aaaaaccf000)
                libm.so.6 => /lib64/libm.so.6 (0x00002aaaab0c6000)
                libomp.so => /g/g92/lin32/opt/romp/pkgs-src/llvm-openmp/openmp/llvm-openmp-install/lib/libomp.so (0x00002aaaab3c8000)
                libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab683000)
                libc.so.6 => /lib64/libc.so.6 (0x00002aaaab89f000)
                libunwind.so.8 => /lib64/libunwind.so.8 (0x00002aaaabc6d000)
                libstdc++.so.6 => /usr/tce/packages/gcc/gcc-6.1.0/lib64/libstdc++.so.6 (0x00002aaaabe87000)
                libgcc_s.so.1 => /usr/tce/packages/gcc/gcc-6.1.0/lib64/libgcc_s.so.1 (0x00002aaaac208000)
                /lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)
                libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaac41e000)

[lin32@quartz188:dataracebench]$ ldd instrumented_app
                linux-vdso.so.1 =>  (0x00002aaaaaacd000)
                libtcmalloc.so.4 => /g/g92/lin32/opt/romp/pkgs-src/gperftools/gperftools-install/lib/libtcmalloc.so.4 (0x00002aaaaaccf000)
                libm.so.6 => /lib64/libm.so.6 (0x00002aaaab0c6000)
                libomp.so => /g/g92/lin32/opt/romp/pkgs-src/llvm-openmp/openmp/llvm-openmp-install/lib/libomp.so (0x00002aaaab3c8000)
                libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab683000)
                libc.so.6 => /lib64/libc.so.6 (0x00002aaaab89f000)
                /g/g92/lin32/opt/romp/pkgs-src/romp-lib/romp-install/lib/libomptrace.so (0x00002aaaabc6d000)
                libdyninstAPI_RT.so => /g/g92/lin32/opt/romp/pkgs-src/dyninst/dyninst-install/lib/libdyninstAPI_RT.so (0x00002aaaac68b000)
                libunwind.so.8 => /lib64/libunwind.so.8 (0x00002aaaad91b000)
                libstdc++.so.6 => /usr/tce/packages/gcc/gcc-6.1.0/lib64/libstdc++.so.6 (0x00002aaaadb35000)
                libgcc_s.so.1 => /usr/tce/packages/gcc/gcc-6.1.0/lib64/libgcc_s.so.1 (0x00002aaaadeb6000)
                /lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)
                libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaae0cc000)
                libelf.so.1 => /lib64/libelf.so.1 (0x00002aaaae2d0000)
                libz.so.1 => /lib64/libz.so.1 (0x00002aaaae4e8000)
Clone this wiki locally