Support for Julia 1.11

[maleadt]

No description provided.

[maleadt]

Encountered this locally; let's see if it happens on CI too:

terminate called after throwing an instance of 'oneapi::mkl::invalid_argument'
  what():  oneapi::mkl::oneapi::mkl::blas::hgemm: invalid argument: Illegal value supplied for parameter lda
  
[1305789] signal 6 (-6): Aborted
in expression starting at /home/tim/Julia/pkg/oneAPI/test/onemkl.jl:1
unknown function (ip: 0x7f0353c84e2c)
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
__verbose_terminate_handler at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++/vterminate.cc:95
__terminate at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:48
terminate at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:58
__cxa_throw at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++/eh_throw.cc:98
_ZN6oneapi3mkl4blasL17check_ld_argumentERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_10MKL_LAYOUTNS0_9transposeElll at /home/tim/.julia/scratchspaces/8f75cd03-7ff8-4ecb-9b8f-daf728133b1b/conda/lib/libmkl_sycl_blas.so.4 (unknown line)
_ZN6oneapi3mkl4blasL21gemm_errchk_argumentsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE10MKL_LAYOUTNS0_9transposeESB_llllll at /home/tim/.julia/scratchspaces/8f75cd03-7ff8-4ecb-9b8f-daf728133b1b/conda/lib/libmkl_sycl_blas.so.4 (unknown line)
_ZN6oneapi3mkl4blas5hgemmERN4sycl3_V15queueE10MKL_LAYOUTNS0_9transposeES7_lllNS0_16value_or_pointerINS3_6detail9half_impl4halfEEEPKSB_lSE_lSC_PSB_lNS1_12compute_modeERKSt6vectorINS3_5eventESaISI_EE at /home/tim/.julia/scratchspaces/8f75cd03-7ff8-4ecb-9b8f-daf728133b1b/conda/lib/libmkl_sycl_blas.so.4 (unknown line)
_ZN6oneapi3mkl4blas12column_major4gemmERN4sycl3_V15queueENS0_9transposeES7_lllNS0_16value_or_pointerINS4_6detail9half_impl4halfEEEPKSB_lSE_lSC_PSB_lNS1_12compute_modeERKSt6vectorINS4_5eventESaISI_EE at /home/tim/.julia/scratchspaces/8f75cd03-7ff8-4ecb-9b8f-daf728133b1b/conda/lib/libmkl_sycl_blas.so.4 (unknown line)
onemklHgemm at /home/tim/.julia/scratchspaces/8f75cd03-7ff8-4ecb-9b8f-daf728133b1b/deps/lib/liboneapi_support.so (unknown line)
onemklHgemm at /home/tim/Julia/pkg/oneAPI/lib/support/liboneapi_support.jl:402
unknown function (ip: 0x7f02c721ff50)
gemm! at /home/tim/Julia/pkg/oneAPI/lib/mkl/wrappers_blas.jl:1212

[maleadt]

The problem is with the Float16 ABI; onemklHgemm is a uint16_t function yet we ccall with Float16, which used to work fine, but in 1.11 we now correctly use the platform Float16 ABI. Problem is that due to switching ABIs, we cannot build a single artifact (using the half ABI) and use that across versions, so I guess we'll have to reinterpret to UInt16 manually on the Julia side.

[maleadt]

I think the easiest way forward is to pass scalars (e.g. alpha/beta inputs) as references instead of values, avoiding any ABI confusion as seen here. This is also what cuBLAS does, and will help us when targeting BFloat16s in the future. I've updated the manual headers and generated wrappers, but I'm not familiar with the generate_interfaces.jl script. @amontoison Could you update it so that all scalars (Ts types in oneMKL) are passed as pointers instead?

On the Julia side, we shouldn't have to do anything; that's the beauty of the Ref type.

[amontoison]

I think the easiest way forward is to pass scalars (e.g. alpha/beta inputs) as references instead of values, avoiding any ABI confusion as seen here. This is also what cuBLAS does, and will help us when targeting BFloat16s in the future. I've updated the manual headers and generated wrappers, but I'm not familiar with the generate_interfaces.jl script. @amontoison Could you update it so that all scalars (Ts types in oneMKL) are passed as pointers instead?

On the Julia side, we shouldn't have to do anything; that's the beauty of the Ref type.

I can do that @maleadt.
I will also document generate_interfaces.jl after #403.

[maleadt]

Thanks. Feel free to push directly to this PR.

[amontoison]

@maleadt We can't pass scalars by reference instead of by value for all oneMKL routines.
This approach works for all BLAS routines except for sdsdot.
I propose not to update LAPACK and SPARSE routines yet, as they currently support only the four main precisions.

[maleadt]

@maleadt We can't pass scalars by reference instead of by value for all oneMKL routines.

Why not? We're talking about the Julia<->Support library interface, these references don't have to be passed to MKL itself.

[amontoison]

Ok, so you want the following thing for all routines, right?

int onemklDgemm(syclQueue_t device_queue, onemklTranspose transa, onemklTranspose transb, int64_t
                m, int64_t n, int64_t k, double *alpha, double *a, int64_t lda, double *b, int64_t ldb,
                double *beta, double *c, int64_t ldc);

extern "C" int onemklDgemm(syclQueue_t device_queue, onemklTranspose transa, onemklTranspose transb,
                           int64_t m, int64_t n, int64_t k, double *alpha, double *a, int64_t lda, double *b,
                           int64_t ldb, double *beta, double *c, int64_t ldc) {
   auto status = oneapi::mkl::blas::column_major::gemm(device_queue->val, convert(transa), convert(transb),
   m, n, k, *alpha, a, lda, b, ldb, *beta, c, ldc, {});
   __FORCE_MKL_FLUSH__(status);
   return 0;
}

[maleadt]

Yes. Otherwise, we rely on the platform/type-dependent argument passing ABIs, which are ill-defined and changing for types like Float16 and BFloat16.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.62%. Comparing base (5cce2db) to head (8e35b4e).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #415      +/-   ##
==========================================
+ Coverage   79.59%   79.62%   +0.03%     
==========================================
  Files          45       45              
  Lines        2524     2523       -1     
==========================================
  Hits         2009     2009              
+ Misses        515      514       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[amontoison]

@maleadt All tests passed 🎉

[maleadt]

Awesome, thanks!

I don't see the updated generate_interfaces.jl though; did you forget to commit that?

[amontoison]

I modified generate_interfaces.jl in the PR #403.
I don't want to push it here because it requires the new format of the headers (with template) that they introduced in oneAPI 2024.1.