Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sigabrt's on Julia 1.9 with native access #51

Closed
JamesWrigley opened this issue Mar 7, 2023 · 5 comments · Fixed by RRZE-HPC/likwid#519
Closed

sigabrt's on Julia 1.9 with native access #51

JamesWrigley opened this issue Mar 7, 2023 · 5 comments · Fixed by RRZE-HPC/likwid#519

Comments

@JamesWrigley
Copy link

Hi there,
I'm having some problems with running LIKWID.jl on my cluster. This is what I've done so far with the native backend:

  • First I tried running @perfmon , it segfaulted because likwid v4.0.0 was installed and the LIKWID.jl bindings were generated against a newer version, so I asked the admins to update it to 5.2.2.

  • Tried running @perfmon again, this time I see a bunch of sigabrts but the Julia process stays alive and the command seemingly completes successfully (though I'm not sure I trust the results). From the stacktraces (see below) I guessed that the issue is this call in likwid to fork(): https://github.com/RRZE-HPC/likwid/blob/v5.2.2/src/access_x86_rdpmc.c#L101. I figured that the sigabrts are coming from the child processes, and that's why the main Julia process doesn't die.

    Test output 
    $ LD_LIBRARY_PATH=/home/wrigleyj/likwid_dist/lib:$LD_LIBRARY_PATH PATH=/home/wrigleyj/likwid_dist/bin:$PATH julia
[ Info: Precompiling AbbreviatedStackTraces [ac637c84-cc71-43bf-9c33-c1b4316be3d4]
[ Info: Skipping precompilation since __precompile__(false). Importing AbbreviatedStackTraces [ac637c84-cc71-43bf-9c33-c1b4316be3d4].
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-beta4 (2023-02-07)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using LIKWID

julia> metrics, events = @perfmon "FLOPS_DP" sum(1:100000)

[47765] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/LibLikwid.jl:838 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:630
unknown function (ip: 0x2b11136079ac)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
perfmon at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:626
unknown function (ip: 0x2b11135fbab6)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
do_call at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
kwcall at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#run_repl#59 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60032.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#1017 at ./client.jl:421
jfptr_YY.1017_52335.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_55341.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
true_main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 8510067 (Pool: 8501372; Big: 8695); GC: 13

[47778] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
segfault_sigaction_rdpmc at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
_L_unlock_13 at /lib64/libpthread.so.0 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/LibLikwid.jl:838 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:630
unknown function (ip: 0x2b11136079ac)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
perfmon at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:626
unknown function (ip: 0x2b11135fbab6)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
do_call at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
kwcall at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#run_repl#59 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60032.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#1017 at ./client.jl:421
jfptr_YY.1017_52335.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_55341.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
true_main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 8510067 (Pool: 8501372; Big: 8695); GC: 13

[47790] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/LibLikwid.jl:838 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:630
unknown function (ip: 0x2b11136079ac)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
perfmon at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:626
unknown function (ip: 0x2b11135fbab6)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
do_call at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
kwcall at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#run_repl#59 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60032.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#1017 at ./client.jl:421
jfptr_YY.1017_52335.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_55341.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
true_main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 8510067 (Pool: 8501372; Big: 8695); GC: 13

[47801] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/LibLikwid.jl:838 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:630
unknown function (ip: 0x2b11136079ac)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
perfmon at /home/wrigleyj/.julia/packages/LIKWID/uvEoZ/src/perfmon.jl:626
unknown function (ip: 0x2b11135fbab6)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
do_call at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
kwcall at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#run_repl#59 at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60032.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
#1017 at ./client.jl:421
jfptr_YY.1017_52335.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_55341.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-beta4+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2689 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/gf.c:2871
jl_apply at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/julia.h:1874 [inlined]
true_main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci4-0/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 8510067 (Pool: 8501372; Big: 8695); GC: 13
ERROR: The selected register PMC0 is in use.
Please run likwid with force option (-f, --force) to overwrite settings

Group: FLOPS_DP
┌──────────────────────────────────┬───────────┐
│                            Event │  Thread 1 │
├──────────────────────────────────┼───────────┤
│                 ACTUAL_CPU_CLOCK │ 1.04286e6 │
│                    MAX_CPU_CLOCK │  828072.0 │
│             RETIRED_INSTRUCTIONS │       NaN │
│              CPU_CLOCKS_UNHALTED │   45161.0 │
│ RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL │       0.0 │
│                            MERGE │       0.0 │
└──────────────────────────────────┴───────────┘
┌──────────────────────┬─────────────┐
│               Metric │    Thread 1 │
├──────────────────────┼─────────────┤
│  Runtime (RDTSC) [s] │  3.66003e-6 │
│ Runtime unhalted [s] │ 0.000434528 │
│          Clock [MHz] │     3022.49 │
│                  CPI │         Inf │
│         DP [MFLOP/s] │         0.0 │
└──────────────────────┴─────────────┘
(OrderedDict("FLOPS_DP" => [OrderedDict("Runtime (RDTSC) [s]" => 3.66002942206152e-6, "Runtime unhalted [s]" => 0.00043452807639303003, "Clock [MHz]" => 3022.4925853523528, "CPI" => Inf, "DP [MFLOP/s]" => 0.0)]), OrderedDict("FLOPS_DP" => [OrderedDict("ACTUAL_CPU_CLOCK" => 1.042859e6, "MAX_CPU_CLOCK" => 828072.0, "RETIRED_INSTRUCTIONS" => 0.0, "CPU_CLOCKS_UNHALTED" => 45161.0, "RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL" => 0.0, "MERGE" => 0.0)]))

  • That code to use rdpmc instructions was added in likwid v5.2.0, so then I tried installing v5.1.1. Unfortunately that segfaults as well because the LIKWID.jl bindings are for a newer version, specifically, this PR changed a struct. I checked the git history of likwid and that struct was changed after rdpmc support was added, so there's no version I can use that will work with LIKWID.jl's binding and doesn't have rdpmc support (and the earliest LIKWID.jl version that doesn't require that particular struct is v0.1.1, so I don't really want to downgrade that...).

TL;DR, @perfmon printed a bunch of sigabrt stacktraces every time I run it because of the way likwid checks for rdpmc support (I think) and I can't downgrade anything to get around it.

Then I tried with the perf_event backend and that kinda worked, modulo some permission errors:

Test with `perf_event` backend 
julia> metrics, events = @perfmon "FLOPS_DP" sum(rand(10000))
ERROR - [./src/includes/perfmon_perfevent.h:perfmon_setupCountersThread_perfevent:894] Permission denied.
Setup of event ACTUAL_CPU_CLOCK on CPU 6 failed: Permission denied
ERROR - [./src/includes/perfmon_perfevent.h:perfmon_setupCountersThread_perfevent:894] Permission denied.
Setup of event MAX_CPU_CLOCK on CPU 6 failed: Permission denied

Group: FLOPS_DP
┌──────────────────────────────────┬──────────┐
│                            Event │ Thread 1 │
├──────────────────────────────────┼──────────┤
│                 ACTUAL_CPU_CLOCK │      0.0 │
│                    MAX_CPU_CLOCK │      0.0 │
│             RETIRED_INSTRUCTIONS │ 100939.0 │
│              CPU_CLOCKS_UNHALTED │  92403.0 │
│ RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL │   1679.0 │
│                            MERGE │      0.0 │
└──────────────────────────────────┴──────────┘
┌──────────────────────┬────────────┐
│               Metric │   Thread 1 │
├──────────────────────┼────────────┤
│  Runtime (RDTSC) [s] │ 9.84015e-5 │
│ Runtime unhalted [s] │        0.0 │
│          Clock [MHz] │        NaN │
│                  CPI │   0.915434 │
│         DP [MFLOP/s] │    17.0627 │
└──────────────────────┴────────────┘
(OrderedDict("FLOPS_DP" => [OrderedDict("Runtime (RDTSC) [s]" => 9.840149344766617e-5, "Runtime unhalted [s]" => 0.0, "Clock [MHz]" => NaN, "CPI" => 0.9154340740447201, "DP [MFLOP/s]" => 17.062749163384993)]), OrderedDict("FLOPS_DP" => [OrderedDict("ACTUAL_CPU_CLOCK" => 0.0, "MAX_CPU_CLOCK" => 0.0, "RETIRED_INSTRUCTIONS" => 100939.0, "CPU_CLOCKS_UNHALTED" => 92403.0, "RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL" => 1679.0, "MERGE" => 0.0)]))

And finally I tried running it on Julia 1.8 and it seemed to work:

Test on Julia 1.8, native backend 
$ LD_LIBRARY_PATH=/home/wrigleyj/likwid_dist/lib:$LD_LIBRARY_PATH PATH=/home/wrigleyj/likwid_dist/bin:$PATH julia +1.8                                                                                  
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.5 (2023-01-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using LIKWID

julia> metrics, events = @perfmon "FLOPS_DP" sum(rand(10000))

Group: FLOPS_DP
┌──────────────────────────────────┬───────────┐
│                            Event │  Thread 1 │
├──────────────────────────────────┼───────────┤
│                 ACTUAL_CPU_CLOCK │ 1.32525e6 │
│                    MAX_CPU_CLOCK │ 1.77396e6 │
│             RETIRED_INSTRUCTIONS │  107060.0 │
│              CPU_CLOCKS_UNHALTED │  158330.0 │
│ RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL │   20239.0 │
│                            MERGE │       0.0 │
└──────────────────────────────────┴───────────┘
┌──────────────────────┬─────────────┐
│               Metric │    Thread 1 │
├──────────────────────┼─────────────┤
│  Runtime (RDTSC) [s] │  8.09407e-5 │
│ Runtime unhalted [s] │ 0.000552193 │
│          Clock [MHz] │     1792.92 │
│                  CPI │     1.47889 │
│         DP [MFLOP/s] │     250.047 │
└──────────────────────┴─────────────┘
(OrderedDict("FLOPS_DP" => [OrderedDict("Runtime (RDTSC) [s]" => 8.094069278487966e-5, "Runtime unhalted [s]" => 0.000552192642979988, "Clock [MHz]" => 1792.9238408240735, "CPI" => 1.478890341864375, "DP [MFLOP/s]" => 250.04727910830042)]), OrderedDict("FLOPS_DP" => [OrderedDict("ACTUAL_CPU_CLOCK" => 1.325251e6, "MAX_CPU_CLOCK" => 1.77396e6, "RETIRED_INSTRUCTIONS" => 107060.0, "CPU_CLOCKS_UNHALTED" => 158330.0, "RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL" => 20239.0, "MERGE" => 0.0)]))

Though sometimes on 1.8 I also see this error, only in Julia and never when running likwid-perfctr:

ERROR: The selected register PMC0 is in use.
Please run likwid with force option (-f, --force) to overwrite settings
@carstenbauer
Copy link
Member

cc @TomTheBear

@JamesWrigley
Copy link
Author

(same thing happens on 1.9-rc1 BTW)

@TomTheBear
Copy link
Collaborator

The error is caused when the cleanup (resetting registers to zero) fails. This may happen if you sigkill an application or it segfaults. If you are alone on the system, just use -f or set environment variable LIKWID_FORCE=1 to overwrite it.

The problem with RDPMC is caused by the child processes used for testing whether the RDPMC instruction can be used with specific offsets. There is currently no switch or environment variable to turn it off. You could deactivate it before compilation by initializing these variables with 0 (https://github.com/RRZE-HPC/likwid/blob/master/src/access_x86_rdpmc.c#L53-L59)

You are on an AMD system and there is the problem that perf_event does not provide access to the fixed-purpose counters (in the AMD world APERF and MPERF) through the normal cpu unit but a separate msr unit. If this unit does not exist, you cannot get the fixed-purpose counters to run. They are used for the Clock metric, so if you don't need this metric, you can ignore the errors. Unfortunately, there are no events for the general-purpose counters (cpu unit) to mimic APERF and MPERF.

@JamesWrigley
Copy link
Author

Thanks for the info! Setting LIKWID_FORCE=1 indeed worked.

I tried changing those RDPMC variables, and it worked the first time, but not the second:

Test output 
$ LD_LIBRARY_PATH=/home/wrigleyj/likwid_dist/lib:$LD_LIBRARY_PATH PATH=/home/wrigleyj/likwid_dist/bin:$PATH LIKWID_FORCE=1 julia
[ Info: Precompiling AbbreviatedStackTraces [ac637c84-cc71-43bf-9c33-c1b4316be3d4]
[ Info: Skipping precompilation since __precompile__(false). Importing AbbreviatedStackTraces [ac637c84-cc71-43bf-9c33-c1b4316be3d4].
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0-rc1 (2023-03-07)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using LIKWID

julia> metrics, events = @perfmon "FLOPS_DP" sum(rand(10000))

Group: FLOPS_DP
┌──────────────────────────────────┬───────────┐
│                            Event │  Thread 1 │
├──────────────────────────────────┼───────────┤
│                 ACTUAL_CPU_CLOCK │ 1.37847e6 │
│                    MAX_CPU_CLOCK │ 1.81805e6 │
│             RETIRED_INSTRUCTIONS │  107331.0 │
│              CPU_CLOCKS_UNHALTED │  155554.0 │
│ RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL │   20239.0 │
│                            MERGE │       0.0 │
└──────────────────────────────────┴───────────┘
┌──────────────────────┬─────────────┐
│               Metric │    Thread 1 │
├──────────────────────┼─────────────┤
│  Runtime (RDTSC) [s] │  9.52408e-5 │
│ Runtime unhalted [s] │ 0.000574368 │
│          Clock [MHz] │      1819.7 │
│                  CPI │     1.44929 │
│         DP [MFLOP/s] │     212.503 │
└──────────────────────┴─────────────┘
(OrderedDict("FLOPS_DP" => [OrderedDict("Runtime (RDTSC) [s]" => 9.524083232550717e-5, "Runtime unhalted [s]" => 0.0005743675194963294, "Clock [MHz]" => 1819.698428178919, "CPI" => 1.4492923759212157, "DP [MFLOP/s]" => 212.50339277620571)]), OrderedDict("FLOPS_DP" => [OrderedDict("ACTUAL_CPU_CLOCK" => 1.37847e6, "MAX_CPU_CLOCK" => 1.818048e6, "RETIRED_INSTRUCTIONS" => 107331.0, "CPU_CLOCKS_UNHALTED" => 155554.0, "RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL" => 20239.0, "MERGE" => 0.0)]))

julia> metrics, events = @perfmon "FLOPS_DP" sum(rand(10000))

[126138] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/LibLikwid.jl:909 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:620
unknown function (ip: 0x2b384391441c)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
perfmon at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:617
unknown function (ip: 0x2b38438faed6)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
do_call at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
start_repl_backend at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#run_repl#59 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60577.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#1017 at ./client.jl:421
jfptr_YY.1017_27809.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_f__call_latest at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_33350.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
true_main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 9095136 (Pool: 9085382; Big: 9754); GC: 14

[126140] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/LibLikwid.jl:909 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:620
unknown function (ip: 0x2b384391441c)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
perfmon at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:617
unknown function (ip: 0x2b38438faed6)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
do_call at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
start_repl_backend at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#run_repl#59 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60577.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#1017 at ./client.jl:421
jfptr_YY.1017_27809.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_f__call_latest at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_33350.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
true_main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 9095136 (Pool: 9085382; Big: 9754); GC: 14

[126141] signal (6.-6): Aborted
in expression starting at REPL[2]:1
gsignal at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
uv_mutex_destroy at /workspace/srcdir/libuv/src/unix/thread.c:429
uv__threadpool_cleanup at /workspace/srcdir/libuv/src/threadpool.c:185
uv_library_shutdown at /workspace/srcdir/libuv/src/uv-common.c:941
_dl_fini at /lib64/ld-linux-x86-64.so.2 (unknown line)
__run_exit_handlers at /lib64/libc.so.6 (unknown line)
exit at /lib64/libc.so.6 (unknown line)
test_rdpmc.constprop.1 at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_x86_rdpmc_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
access_client_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
HPMaddThread at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init_maps at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/likwid_dist/lib/liblikwid.so (unknown line)
perfmon_init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/LibLikwid.jl:909 [inlined]
init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:35
init at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:46 [inlined]
#perfmon#9 at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:620
unknown function (ip: 0x2b384391441c)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
perfmon at /home/wrigleyj/.julia/packages/LIKWID/FGtXJ/src/perfmon.jl:617
unknown function (ip: 0x2b38438faed6)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
do_call at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
jl_toplevel_eval_flex at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
eval_user_input at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
repl_backend_loop at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
#start_repl_backend#46 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
start_repl_backend at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:231
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#run_repl#59 at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:377
run_repl at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/usr/share/julia/stdlib/v1.9/REPL/src/REPL.jl:363
jfptr_run_repl_60577.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
#1017 at ./client.jl:421
jfptr_YY.1017_27809.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
jl_f__call_latest at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:816 [inlined]
invokelatest at ./essentials.jl:813 [inlined]
run_main_repl at ./client.jl:405
exec_options at ./client.jl:322
_start at ./client.jl:522
jfptr__start_33350.clone_1 at /home/wrigleyj/.julia/juliaup/julia-1.9.0-rc1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2731 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/gf.c:2913
jl_apply at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/julia.h:1878 [inlined]
true_main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci5-5/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 9095136 (Pool: 9085382; Big: 9754); GC: 14

Group: FLOPS_DP
┌──────────────────────────────────┬───────────┐
│                            Event │  Thread 1 │
├──────────────────────────────────┼───────────┤
│                 ACTUAL_CPU_CLOCK │ 1.51049e6 │
│                    MAX_CPU_CLOCK │ 1.25016e6 │
│             RETIRED_INSTRUCTIONS │  105946.0 │
│              CPU_CLOCKS_UNHALTED │  189687.0 │
│ RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL │   20239.0 │
│                            MERGE │       0.0 │
└──────────────────────────────────┴───────────┘
┌──────────────────────┬─────────────┐
│               Metric │    Thread 1 │
├──────────────────────┼─────────────┤
│  Runtime (RDTSC) [s] │  8.20407e-5 │
│ Runtime unhalted [s] │ 0.000629378 │
│          Clock [MHz] │     2899.75 │
│                  CPI │     1.79041 │
│         DP [MFLOP/s] │     246.695 │
└──────────────────────┴─────────────┘
(OrderedDict("FLOPS_DP" => [OrderedDict("Runtime (RDTSC) [s]" => 8.204071696749904e-5, "Runtime unhalted [s]" => 0.0006293775835720538, "Clock [MHz]" => 2899.7500471154403, "CPI" => 1.790412096728522, "DP [MFLOP/s]" => 246.6945773769604)]), OrderedDict("FLOPS_DP" => [OrderedDict("ACTUAL_CPU_CLOCK" => 1.510493e6, "MAX_CPU_CLOCK" => 1.25016e6, "RETIRED_INSTRUCTIONS" => 105946.0, "CPU_CLOCKS_UNHALTED" => 189687.0, "RETIRED_SSE_AVX_FLOPS_DOUBLE_ALL" => 20239.0, "MERGE" => 0.0)]))

But I assume there's something else I would need to change to fully disable it. Would you be open to a PR for that?

You are on an AMD system and there is the problem that perf_event does not provide access to the fixed-purpose counters (in the AMD world APERF and MPERF) through the normal cpu unit but a separate msr unit. If this unit does not exist, you cannot get the fixed-purpose counters to run. They are used for the Clock metric, so if you don't need this metric, you can ignore the errors. Unfortunately, there are no events for the general-purpose counters (cpu unit) to mimic APERF and MPERF.

Thanks, indeed the perf_event backend on an Intel machine works (though not with the CACHE group, but it's something at least).

@TomTheBear
Copy link
Collaborator

PRs are always welcomed. I just checked whether LIKWID checks the "RDPMC allowed" bit with CPUID and it does, so I'm not sure how to better detect it without the child processes.

It seems that @perfmon is doing LIKWID's init and finalize and I forgot to send you the line numbers for the finalize part where the variables are set to -1 again: https://github.com/RRZE-HPC/likwid/blob/master/src/access_x86_rdpmc.c#L230-L236

The CACHE group is simply not available for all platforms. It can be created for almost all (also by yourself) but I don't like the group. It has a huge overhead compared to separate L2+L3+MEM.

@JamesWrigley JamesWrigley mentioned this issue Mar 8, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use _exit() instead of exit() when checking for RDPMC support JamesWrigley/likwid
3 participants
@carstenbauer @TomTheBear @JamesWrigley