[Experimental CPU runtime] Clashes with system libraries

[inducer]

I would very much like to use the Intel CPU ICD, but it is (IMO) unnecessarily logistically difficult for me to do so, because the runtime ships with multiple shared objects that clash with system libraries, including libtbb.so.2 and libOpenCL.so.2. If I use the Intel CPU runtime without configuring the dynamic linker path to include/prefer the path of the Intel CPU CL runtime, then I get nothing but crashes (segfaults) out of the runtime, since the system-built libraries are ostensibly different from the ones shipped with the CL runtime. On the other hand, if I set (e.g.) $LD_LIBRARY_PATH to the path to the runtime, the runtime works as intended, but now everything on my system uses the (presumably slightly incompatible) libraries shipped with the runtime.

Would it be possible to build binaries of the runtime with shared libraries that do not clash with other shared objects present on a standard Linux system? Thanks!

[inducer]

Experimental results on Haswell, running the PyOpenCL test suite:

Without setting $LD_LIBRARY_PATH:

PYOPENCL_TEST=int python -m pytest                                                                                                                                                                                                                                                               andreask_work@koelsch 19:56
=================================================================================================================================================================== test session starts ===================================================================================================================================================================
platform linux -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /home/andreask_work/src/pyopencl
collected 210 items                                                                                                                                                                                                                                                                                                                                       

test_algorithm.py ......................................................                                                                                                                                                                                                                                                                            [ 25%]
test_array.py ...................[1]    30040 segmentation fault (core dumped)  PYOPENCL_TEST=int python -m pytest

With setting $LD_LIBRARY_PATH:

=================================================================================================================================================================== test session starts ===================================================================================================================================================================
platform linux -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0                                                                                                                                                                                                                                                                                      
rootdir: /home/andreask_work/src/pyopencl                                                                                                                                                                                                                                                                                                                  
collected 210 items                                                                                                                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                                                                                                           
test_algorithm.py ......................................................                                                                                                                                                                                                                                                                            [ 25%]
test_array.py ........................................................................                                                                                                                                                                                                                                                              [ 60%]
test_clmath.py ............................                                                                                                                                                                                                                                                                                                         [ 73%] 
test_clrandom.py .....................                                                                                                                                                                                                                                                                                                              [ 83%]
test_enqueue_copy.py ..                                                                                                                                                                                                                                                                                                                             [ 84%] 
test_wrapper.py ........s.........s..............                                                                                                                                                                                                                                                                                                   [100%] 
                                                     
(snip)
================================================================================================================================================== 208 passed, 2 skipped, 194 warnings in 94.96 seconds ===================================================================================================================================================

[alexbatashev]

@inducer I'm afraid I didn't get you. Does setting $LD_LIBRARY_PATH cause other significant problems for you? You see, this is how this experimental runtime is intended to work. You set an env var and go. You can add a bash alias to speed thing up. However, you might want to try slight modifications:

1. Place runtime to /opt/intel/expcpu or whatever location you feel good.
2. Remove libOpenCL and symlinks from RT'slib directory.
3. Create a new file (let's say intel_expcpu.icd) in /etc/OpenCL/vendors/ and put there libintelocl.so.
4. Export path to RT to LD_LIBRARY_PATH in your bashrc.

If you wish to do so, I strongly recommend you using latest and greatest Khronos ICD. You see, AFAIK Intel is the only vendor that ships OpenCL 2.1. Other implementations may be incompatible. On the other hand, vanilla ICD may break other drivers if they use some proprietary APIs (highly unlikely, though).

[inducer]

What I'm meaning to suggest is that you rename your libtbb.so.2 to libtbb-intel-ocl.so.2 (or so) in your shipped binaries. Since this library seems (see evidence above) to be incompatible with the libtbb.so.2 already present on my system, this will have two (IMO) positive effects:

• The Intel CL runtime will not accidentally use the system library (and thus avoid the crashes I mention above).
• The rest of my system will not accidentally use the libtbb.so.2 shipped with the CL runtime.

(and similarly for all other shared library names for which there is potential for conflict with system libraries)

[alexbatashev]

@inducer ok, I see. Fair point, because on Windows libtbb.so is actually ocltbb64.dll. They probably should be aligned, I'll ask OpenCL team to have a look.

[alexbatashev]

@inducer It is not advised to rename libtbb.so library. By default each shared library on Linux exports all symbols. If an application uses both TBB and OpenCL, renamed library may cause ambiguity. This leads to stability issues for many applications. Furthermore, two thread pools are created, implying performance degradation. However, TBB team maintains backward compatibility. This means that you can replace an old TBB library with a newer one without problems. So, as a workaround I suggest you updating you system-wide TBB library to the latest version and remove OpenCL libtbb.so from your LD_LIBRARY_PATH.

[inducer]

Thanks for your advice! Unfortunately, your suggested solution doesn't quite work for me. I have a Debian-packaged version of TBB 2019 Update 8 on my system, however it appears that that is incompatible with the CL runtime binary in a way that the shipped TBB is not.

[alexbatashev]

@inducer can you please provide us with more information then? Please, run your OpenCL app like this

LD_DEBUG=libs ./my-app > log.txt 2>&1

and share your log file with us.

[inducer]

• PyOpenCL Intel CL dynamic linker debug log with LD_LIBRARY_PATH_SET, succeeds
• PyOpenCL Intel CL dynamic linker debug log without LD_LIBRARY_PATH_SET, crashes

[alexbatashev]

@inducer from what I can see in logs, the main difference between two runs is that when LD_LIBRARY_PATH is set, you use OpenCL ICD (Installable Client Driver) from Intel experimental OpenCL CPU Runtime. A great explanation of how these things work can be found here https://bashbaug.github.io/opencl/2019/07/06/OpenCL-On-Linux.html. I'm not sure about the origin of your system-wide ICD, but I suppose it is a bit outdated. This may cause some compatibility problems. If I were you, I'd try updating it to latest version. You cat search in apt repositories for a newer version (apt-cache search ocl-icd should list you available packages). Alternatively, you can download latest version from Khronos GitHub page https://github.com/KhronosGroup/OpenCL-ICD-Loader and build manually. I hope this helps you with your issue.

[alexbatashev]

@inducer can you please confirm/deny your issue is resolved?

[inducer]

Could you point me to the source code for the runtime? (I can't seem to find it in the source tarball distributed along the binary release.) Without that, it's hard for me to determine whether my ICD loader (https://github.com/OCL-dev/ocl-icd, latest version, default in Debian) is doing something wrong.

[alexbatashev]

Could you point me to the source code for the runtime? (I can't seem to find it in the source tarball distributed along the binary release.) Without that, it's hard for me to determine whether my ICD loader (https://github.com/OCL-dev/ocl-icd, latest version, default in Debian) is doing something wrong.

The source code is proprietary, you can’t get access to it.

[alexbatashev]

@inducer if you still experience problems, please reopen this issue.