Could not load library "liblikwid"

[Cvikli]

Hey,
The tool video looked amaizing. Exactly what I need.
I tried to install and run it with the CPU version, but I stucked at:

ERROR: LoadError: TaskFailedException
Stacktrace:
 [1] wait
   @ ./task.jl:334 [inlined]
 [2] threading_run(func::Function)
   @ Base.Threads ./threadingconstructs.jl:38
 [3] macro expansion
   @ ./threadingconstructs.jl:97 [inlined]
 [4] get_processor_ids()
   @ LIKWID ~/.julia/packages/LIKWID/EYAWk/src/affinity.jl:105
 [5] perfmon(f::Function, group_or_groups::String)
   @ LIKWID.PerfMon ~/.julia/packages/LIKWID/EYAWk/src/perfmon.jl:618
 [6] top-level scope
   @ ~/.julia/packages/LIKWID/EYAWk/src/perfmon.jl:707

    nested task error: could not load library "liblikwid"
    liblikwid.so: cannot open shared object file: No such file or directory
    Stacktrace:
     [1] likwid_getProcessorId
       @ ~/.julia/packages/LIKWID/EYAWk/src/LibLikwid.jl:577 [inlined]
     [2] get_processor_id
       @ ~/.julia/packages/LIKWID/EYAWk/src/affinity.jl:76 [inlined]
     [3] macro expansion
       @ ~/.julia/packages/LIKWID/EYAWk/src/affinity.jl:106 [inlined]
     [4] (::LIKWID.var"#119#threadsfor_fun#6"{Vector{Int64}, UnitRange{Int64}})(onethread::Bool)
       @ LIKWID ./threadingconstructs.jl:85
     [5] (::LIKWID.var"#119#threadsfor_fun#6"{Vector{Int64}, UnitRange{Int64}})()
       @ LIKWID ./threadingconstructs.jl:52
in expression starting at /home/master/repos/tests/speedtests/test_cpu_likwid.jl:16

I installed Likwid: I think I did exactly like it is mentioned in the video:

# Path were to install likwid
PREFIX ?= /home/master/.local#NO SPACE

# Set the default mode for MSR access.
# This can usually be overriden on the commandline.
# Valid values are: direct, accessdaemon and perf_event
ACCESSMODE = accessdaemon#NO SPACE

...
...

# Build LIKWID with NVIDIA interface (CUDA, CUPTI)
# For configuring include paths, go to CUDA section
NVIDIA_INTERFACE = false#NO SPACE

added: ] add LIKWID

(Also tried with this:
PREFIX ?= /usr/local#NO SPACE)

Both arrives to the same result that I missing the library. Should I load it explicitly with julia or what is this?

[TomTheBear]

Hi,

It's just a guess but I think the problem is that neither /usr/local/lib nor /home/master/.local/lib are in your LD_LIBRARY_PATH. Add it (export LD_LIBRARY_PATH=<LIKWID_LIB_DIR>:$LD_LIBRARY_PATH) and try again.

Best,
Thomas

[Cvikli]

Hi, yes.
Where is my lib directory, I just couldn't locate it even after installed LIKWID.
Best,
Marcell

[TomTheBear]

When you install LIKWID, you specify the installation path in the PREFIX variable. After you do a make distclean && make && sudo make install, there should be a PREFIX/lib directory containing the LIKWID library. This path needs to be in LD_LIBRARY_PATH.

[Cvikli]

I still get this:
could not load library "liblikwid"
The lib is there for sure. It should see.

Shouldn't I add some flag to the julia compiler? I think it basically doesn't even load the library? But it isn't mentioned anywhere so I didn't try it yet to set it explicitedly to load...

[Cvikli]

I see, there is no flag to load library explicitedly for julia. :( I thought it will work like GCC... :D
I don't understand then why it just doesn't see it.

Listing my dir: /usr/local/lib:

clang                             libclangTesting.a               libLLVMAMDGPUAsmParser.a     libLLVMDebugInfoDWARF.a       libLLVMLibDriver.a            libLLVMRemarks.a                  libLLVMWindowsManifest.a
cmake                             libclangTooling.a               libLLVMAMDGPUCodeGen.a       libLLVMDebugInfoGSYM.a        libLLVMLineEditor.a           libLLVMRISCVAsmParser.a           libLLVMX86AsmParser.a
libclangAnalysis.a                libclangToolingASTDiff.a        libLLVMAMDGPUDesc.a          libLLVMDebugInfoMSF.a         libLLVMLinker.a               libLLVMRISCVCodeGen.a             libLLVMX86CodeGen.a
libclangAnalysisFlowSensitive.a   libclangToolingCore.a           libLLVMAMDGPUDisassembler.a  libLLVMDebugInfoPDB.a         libLLVMLTO.a                  libLLVMRISCVDesc.a                libLLVMX86Desc.a
libclangAPINotes.a                libclangToolingInclusions.a     libLLVMAMDGPUInfo.a          libLLVMDemangle.a             libLLVMMC.a                   libLLVMRISCVDisassembler.a        libLLVMX86Disassembler.a
libclangARCMigrate.a              libclangToolingRefactoring.a    libLLVMAMDGPUTargetMCA.a     libLLVMDiff.a                 libLLVMMCA.a                  libLLVMRISCVInfo.a                libLLVMX86Info.a
libclangAST.a                     libclangToolingSyntax.a         libLLVMAMDGPUUtils.a         libLLVMDlltoolDriver.a        libLLVMMCDisassembler.a       libLLVMRuntimeDyld.a              libLLVMXCoreCodeGen.a
libclangASTMatchers.a             libclangTransformer.a           libLLVMAnalysis.a            libLLVMDWARFLinker.a          libLLVMMCJIT.a                libLLVMScalarOpts.a               libLLVMXCoreDesc.a
libclangBasic.a                   libear                          libLLVMARMAsmParser.a        libLLVMDWP.a                  libLLVMMCParser.a             libLLVMSelectionDAG.a             libLLVMXCoreDisassembler.a
libclangCodeGen.a                 libgmock.a                      libLLVMARMCodeGen.a          libLLVMExecutionEngine.a      libLLVMMipsAsmParser.a        libLLVMSparcAsmParser.a           libLLVMXCoreInfo.a
libclang-cpp.so                   libgmock_main.a                 libLLVMARMDesc.a             libLLVMExegesis.a             libLLVMMipsCodeGen.a          libLLVMSparcCodeGen.a             libLLVMXRay.a
libclang-cpp.so.14git             libgtest.a                      libLLVMARMDisassembler.a     libLLVMExegesisAArch64.a      libLLVMMipsDesc.a             libLLVMSparcDesc.a                libLTO.so
libclangCrossTU.a                 libgtest_main.a                 libLLVMARMInfo.a             libLLVMExegesisMips.a         libLLVMMipsDisassembler.a     libLLVMSparcDisassembler.a        libLTO.so.14git
libclangDependencyScanning.a      liblikwid-hwloc.so              libLLVMARMUtils.a            libLLVMExegesisPowerPC.a      libLLVMMipsInfo.a             libLLVMSparcInfo.a                libpng16.a
libclangDirectoryWatcher.a        liblikwid-hwloc.so.5            libLLVMAsmParser.a           libLLVMExegesisX86.a          libLLVMMIRParser.a            libLLVMSupport.a                  libpng16.la
libclangDriver.a                  liblikwid-hwloc.so.5.2          libLLVMAsmPrinter.a          libLLVMExtensions.a           libLLVMMSP430AsmParser.a      libLLVMSymbolize.a                libpng16.so
libclangDynamicASTMatchers.a      liblikwid-lua.so                libLLVMAVRAsmParser.a        libLLVMFileCheck.a            libLLVMMSP430CodeGen.a        libLLVMSystemZAsmParser.a         libpng16.so.16
libclangEdit.a                    liblikwid-lua.so.5              libLLVMAVRCodeGen.a          libLLVMFrontendOpenACC.a      libLLVMMSP430Desc.a           libLLVMSystemZCodeGen.a           libpng16.so.16.37.0
libclangFormat.a                  liblikwid-lua.so.5.2            libLLVMAVRDesc.a             libLLVMFrontendOpenMP.a       libLLVMMSP430Disassembler.a   libLLVMSystemZDesc.a              libpng.a
libclangFrontend.a                liblikwidpin.so                 libLLVMAVRDisassembler.a     libLLVMFuzzMutate.a           libLLVMMSP430Info.a           libLLVMSystemZDisassembler.a      libpng.la
libclangFrontendTool.a            liblikwidpin.so.5               libLLVMAVRInfo.a             libLLVMGlobalISel.a           libLLVMNVPTXCodeGen.a         libLLVMSystemZInfo.a              libpng.so
libclangHandleCXX.a               liblikwidpin.so.5.2             libLLVMBinaryFormat.a        libLLVMHexagonAsmParser.a     libLLVMNVPTXDesc.a            libLLVMTableGen.a                 libRemarks.so
libclangHandleLLVM.a              liblikwid.so                    libLLVMBitReader.a           libLLVMHexagonCodeGen.a       libLLVMNVPTXInfo.a            libLLVMTableGenGlobalISel.a       libRemarks.so.14git
libclangIndex.a                   liblikwid.so.5                  libLLVMBitstreamReader.a     libLLVMHexagonDesc.a          libLLVMObjCARCOpts.a          libLLVMTarget.a                   libscanbuild
libclangIndexSerialization.a      liblikwid.so.5.2                libLLVMBitWriter.a           libLLVMHexagonDisassembler.a  libLLVMObject.a               libLLVMTextAPI.a                  LLVMPromotePointerKernArgsToGlobal.so
libclangInterpreter.a             liblldCOFF.a                    libLLVMBPFAsmParser.a        libLLVMHexagonInfo.a          libLLVMObjectYAML.a           libLLVMTransformUtils.a           LLVMSelectAcceleratorCode.so
libclangLex.a                     liblldCommon.a                  libLLVMBPFCodeGen.a          libLLVMInstCombine.a          libLLVMOffloadArch.a          libLLVMVEAsmParser.a              node_modules
libclangParse.a                   liblldELF.a                     libLLVMBPFDesc.a             libLLVMInstrumentation.a      libLLVMOption.a               libLLVMVECodeGen.a                ocaml
libclangRewrite.a                 liblldMachO.a                   libLLVMBPFDisassembler.a     libLLVMInterfaceStub.a        libLLVMOrcJIT.a               libLLVMVectorize.a                pkgconfig
libclangRewriteFrontend.a         liblldMinGW.a                   libLLVMBPFInfo.a             libLLVMInterpreter.a          libLLVMOrcShared.a            libLLVMVEDesc.a                   python2.7
libclangSema.a                    liblldWasm.a                    libLLVMCFGuard.a             libLLVMipo.a                  libLLVMOrcTargetProcess.a     libLLVMVEDisassembler.a           python3.6
libclangSerialization.a           libLLVMAArch64AsmParser.a       libLLVMCFIVerify.a           libLLVMIRReader.a             libLLVMPasses.a               libLLVMVEInfo.a                   python3.8
libclang.so                       libLLVMAArch64CodeGen.a         libLLVMCodeGen.a             libLLVMJITLink.a              libLLVMPowerPCAsmParser.a     libLLVMWebAssemblyAsmParser.a     python3.9
libclang.so.13                    libLLVMAArch64Desc.a            libLLVMCore.a                libLLVMLanaiAsmParser.a       libLLVMPowerPCCodeGen.a       libLLVMWebAssemblyCodeGen.a
libclang.so.14.0.0git             libLLVMAArch64Disassembler.a    libLLVMCoroutines.a          libLLVMLanaiCodeGen.a         libLLVMPowerPCDesc.a          libLLVMWebAssemblyDesc.a
libclangStaticAnalyzerCheckers.a  libLLVMAArch64Info.a            libLLVMCoverage.a            libLLVMLanaiDesc.a            libLLVMPowerPCDisassembler.a  libLLVMWebAssemblyDisassembler.a
libclangStaticAnalyzerCore.a      libLLVMAArch64Utils.a           libLLVMDebugInfoCodeView.a   libLLVMLanaiDisassembler.a    libLLVMPowerPCInfo.a          libLLVMWebAssemblyInfo.a
libclangStaticAnalyzerFrontend.a  libLLVMAggressiveInstCombine.a  libLLVMDebuginfod.a          libLLVMLanaiInfo.a            libLLVMProfileData.a          libLLVMWebAssemblyUtils.a

In sort:

➜  /usr/local/lib ls liblikwid*
liblikwid-hwloc.so  liblikwid-hwloc.so.5  liblikwid-hwloc.so.5.2  liblikwid-lua.so  liblikwid-lua.so.5  liblikwid-lua.so.5.2  liblikwidpin.so  liblikwidpin.so.5  liblikwidpin.so.5.2  liblikwid.so  liblikwid.so.5  liblikwid.so.5.2

[TomTheBear]

As far as I know the library is dynamically loaded. My Julia knowledge is limited, so we better wait for wise people.

[Cvikli]

Ok, I think the problem was:

I had to "Quit" from vscode and not just reload window...
Sorry for the hassle.
Also thank you for your help!

[Cvikli]

I think in the doc there should be some note on the LD_LIBRARY_PATH and also not to just "restart the julia language server" and "reload the vscode window", so others don't run into this issue. :D
Instead add to LD_LIBRARY_PATH the path you mentioned and do a whole extensive restart of the vscode.

[carstenbauer]

Yeah mentioning this could be helpful. Mind opening a PR? I'm thinking of adding a little "installing likwid" section that explains the basics and links to the likwid wiki for more details.

[carstenbauer]

Also, once we have likwid as a Julia artifact this should be the simplest way to get likwid for likwid.jl (although one doesn't get all if the features)

[Cvikli]

I cannot make the PR as I am too short on time due to our company, sorry for that.

Install section could be updated that is all. But really great work and will try to use LIKWID.jl as much as possible. It worked on test cases for now. :)