New package: intel-compute-runtime-20.21.16886

[st3r4g]

Closes #11364

Resumed the work I did some time ago, mainly updated the templates to use llvm10. Opening this as a draft for now because I remember musl requiring to be patched (I gave up back then), any help is welcome.

@ericonr

[ahesford]

I am not able to fully test because intel-compute-runtime appears to have a hard build dependency on libexecinfo, which is currently marked for musl only. I've flagged a few style issues and a build issue in intel-graphics-compiler that can be remedied to make that package build.

[st3r4g]

@ahesford this should now build on x86-64, feel free to test.

As for musl, not sure what is the correct way to fix this:

/builddir/compute-runtime-20.18.16699/shared/source/os_interface/linux/os_library_linux.cpp: In constructor 'NEO::Linux::OsLibrary::OsLibrary(const string&)':
/builddir/compute-runtime-20.18.16699/shared/source/os_interface/linux/os_library_linux.cpp:35:49: error: 'RTLD_DEEPBIND' was not declared in this scope
   35 |         constexpr auto dlopenFlag = RTLD_LAZY | RTLD_DEEPBIND;
      |                                                 ^~~~~~~~~~~~~

[ericonr]

It should come from dlfcn.h, which, on musl, doesn't seem to define that type. /usr/include/python3.8/pyconfig.h seems to have some tricks for dealing with that, so I guess we can try to think of a patch based on that.

The intel use case for that option is mentioned here as well https://stackoverflow.com/questions/34073051/when-we-are-supposed-to-use-rtld-deepbind#34074587

[ericonr]

intel/compute-runtime#266 suggests passing a SANITIZER_BUILD define to the CXXFLAGS. Makes sense, this is the only place where deepbind is used: https://github.com/intel/compute-runtime/blob/ee5ebde417716c8cacfbc42c0c7cd840339188d9/shared/source/os_interface/linux/os_library_linux.cpp#L32

As a more correct fix, however, intel/compute-runtime#265 (comment) recommends using RTLD_GLOBAL instead of DEEPBIND. Upstream seems quite responsive, we could look into communicating with them if any more issues appear.

[st3r4g]

@ericonr thanks! didn't see there has been some work from alpine since last time I tried.
Indeed the combination of SANITIZER_BUILD + RTLD_GLOBAL fixes the error I was hitting soon after with the generation of the kernels

[st3r4g]

Some tests are failing on musl:

[ 93%] Built target run_kbl_unit_tests
Scanning dependencies of target run_unit_tests
running tests for linux dynamic library - .so in /builddir/compute-runtime-20.18.16699/build/bin
Iteration: 1
Iteration: 1. random_seed: 0

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/drm_null_device_tests.cpp:28: Failure
Expected: (drmNullDevice) != (nullptr), actual: NULL vs (nullptr)
[  FAILED  ][ --- ][ 0 ] DrmNullDeviceTests.GIVENdrmNullDeviceWHENcallGetDeviceIdTHENreturnProperDeviceId

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/drm_null_device_tests.cpp:28: Failure
Expected: (drmNullDevice) != (nullptr), actual: NULL vs (nullptr)
[  FAILED  ][ --- ][ 0 ] DrmNullDeviceTests.GIVENdrmNullDeviceWHENcallIoctlTHENalwaysSuccess

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/drm_null_device_tests.cpp:28: Failure
Expected: (drmNullDevice) != (nullptr), actual: NULL vs (nullptr)
[  FAILED  ][ --- ][ 0 ] DrmNullDeviceTests.GIVENdrmNullDeviceWHENregReadOtherThenTimestampReadTHENalwaysSuccess

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/drm_null_device_tests.cpp:28: Failure
Expected: (drmNullDevice) != (nullptr), actual: NULL vs (nullptr)
[  FAILED  ][ --- ][ 0 ] DrmNullDeviceTests.GIVENdrmNullDeviceWHENgetGpuTimestamp32bOr64bTHENerror

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/drm_null_device_tests.cpp:28: Failure
Expected: (drmNullDevice) != (nullptr), actual: NULL vs (nullptr)
[  FAILED  ][ --- ][ 0 ] DrmNullDeviceTests.GIVENdrmNullDeviceWHENgetGpuTimestamp36bTHENproperValues

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/main_linux_dll.cpp:81: Failure
Expected equality of these values:
  2u
    Which is: 2
  hwDeviceIds.size()
    Which is: 0
[  FAILED  ][ --- ][ 0 ] DrmTest.GivenTwoOpenableDevicesWhenDiscoverDevicesThenCreateTwoHwDeviceIds

/builddir/compute-runtime-20.18.16699/opencl/test/unit_test/linux/main_linux_dll.cpp:103: Failure
Expected equality of these values:
  1u
    Which is: 1
  hwDeviceIds.size()
    Which is: 0
make[2]: *** [opencl/test/unit_test/CMakeFiles/run_unit_tests.dir/build.make:78: run_unit_tests] Segmentation fault
make[1]: *** [CMakeFiles/Makefile2:8180: opencl/test/unit_test/CMakeFiles/run_unit_tests.dir/all] Error 2
make: *** [Makefile:172: all] Error 2

also note that it ends with a segfault. We have to find out if really the compute-runtime is broken or just these tests.

[st3r4g]

Needs to be tested with real world CL programs, especially on musl

[ericonr]

I'm doing some quick testing on Darktable (on glibc), seems to be working fine. I will try to test darktable CLI on musl afterwards.

EDIT: darktable simple image worked! Both on glibc and musl, I tested an image with all effects enabled, and intel_gpu_top showed high usage in the Render line, sometimes at 100% even. No crashes or segfaults.

[ericonr]

Btw, I believe you're missing some additions from common/shlibs (at least xbps-src complained about that here). I can get them for you if you want.

[ericonr]

The patches that you're using, do they come from upstream? At least the main repo seemed open to solving musl problems, so we can probably get all that upstreamed in the future c:

Great work btw, thanks!

[st3r4g]

Btw, I believe you're missing some additions from common/shlibs (at least xbps-src complained about that here). I can get them for you if you want.

I think the usual policy is to only add those actually required by other packages

[ericonr]

https://github.com/intel/intel-graphics-compiler/releases we have a new release!

[st3r4g]

Updated

[ericonr]

They didn't include the CMake patch in the release :c

[st3r4g]

Oof I thought they did. I have to readd it back.

[ericonr]

https://github.com/intel/compute-runtime/releases/tag/20.21.16886

https://github.com/intel/intel-graphics-compiler/releases/tag/igc-1.0.4053 <- can remove the CMake patch

https://github.com/intel/opencl-clang/releases/tag/v10.0.0-2 <- failing, might have to file upstream bug

[ericonr]

Working for darktable on glibc and musl. Updated runtime and compiler, but not opencl-clang.

[the-maldridge]

@ericonr what is the status of this PR?

[ericonr]

On my to-do list to update and merge.

If @st3r4g is up to updating, that'd be great as well.

[st3r4g]

Lost interest a bit since I'm not actually using it atm, but I might resume it some day, if ericonr doesn't do it before.
The status was: apparently working on musl, but some tests were failing.

[ericonr]

Not surprised this is weirdly broken, even with patches for llvm12 compat.

Branch: https://github.com/ericonr/void-packages/tree/intel-comp

Build failure: https://gist.github.com/033f9a3d0279de04c7a1db7a9b1fd70a

[github-actions]

Pull Requests become stale 90 days after last activity and are closed 14 days after that. If this pull request is still relevant bump it or assign it.