SGEMM broken with 1.6.2 on Intel ARC

[0cc4m]

output_1.6.1.txt
output_1.6.2.txt

In 1.6.1 SGEMM gives the correct results on Intel ARC (tested on an A770 on Arch Linux), but with 1.6.2 the results are wrong.

Let me know if more data is needed.

[CNugteren]

This is most likely because of #503, which added new tuning results for the A770, as shared here. Apparently they are not (always?) valid, or there is a bug in either of your A770 drivers.
I think the best path forward is to remove those tuning results for now, and then later optionally re-tune on @0cc4m's set-up and see if that leads to errors as well.

[tangjinchuan]

Dear Cedric,
I have put the tuning results based on my A770 card today for you to look at or update.
#1 (comment)

[CNugteren]

I have put the tuning results based on my A770 card today for you to look at or update.

Thanks, but I don't have an A770 to test it. Could you or @0cc4m apply those tuning results, re-compile the library, and then run the tests?

[tangjinchuan]

I suggest @0cc4m give it a try to check urgent test cases.

[0cc4m]

No, that tuning doesn't work for me. I even tuned it myself and it still fails the tests. It seems the Linux driver (mesa version 24.0.1) for A770 has some issue that means the results are wrong when CLBlast is tuned. Very odd.

[tangjinchuan]

I see. Please try installing intel compute runtime if this is not the case. The tuning is optimized for the Intel NEO Platform, not the open-source Mesa Platform.
https://github.com/intel/compute-runtime
The command would be similar to
apt-get install intel-opencl-icd
Also, please provide clinfo if the problem persists.
Intel team told me both Linux and Windows used the same codebase (https://community.intel.com/t5/GPU-Compute-Software/Driver-problem-with-OpenCL-kernel/m-p/1362525), hence the tuning should work properly.
Let's see if any goodness comes out of this.

[0cc4m]

@tangjinchuan Sorry, my bad. Somehow I had it in my head that the Intel OpenCL driver is in mesa, but of course it's separate. I'm using intel-compute-runtime 23.48.27912.11-1.

[tangjinchuan]

@0cc4m Please give the latest version [24.05.28454.6] a go: https://github.com/intel/compute-runtime/releases, all the install steps are there. If the problem persists try turning the PC on and off again for I once had such a problem after updating in Windows. Meanwhile, it would be very helpful to have a test case on two matrices multiplication (just give two matrices that can have such a problem) so that I can see if it produces the same problems on my card. Locating problems in LLM inference framework would be too much for me.
By the way, thanks for your contributions to llama.cpp.

[0cc4m]

@0cc4m Please give the latest version [24.05.28454.6] a go: https://github.com/intel/compute-runtime/releases, all the install steps are there. If the problem persists try turning the PC on and off again for I once had such a problem after updating in Windows. Meanwhile, it would be very helpful to have a test case on two matrices multiplication (just give two matrices that can have such a problem) so that I can see if it produces the same problems on my card. Locating problems in LLM inference framework would be too much for me. By the way, thanks for your contributions to llama.cpp.

I'm not using Windows, the issue is persistent across reboots and I'm not using llama.cpp to test it. I'm just running clblast_test_xgemm as described in how to test the CLBlast library for correctness.

It still fails on intel-compute-runtime version 24.05.28454.6 once tuned for A770. It works fine if not tuned.

[tangjinchuan]

Thanks! Will have the test case a try next day. In the meantime, for any windows users, you can also give it a go.
clblast_test_xgemm.zip

[tangjinchuan]

@CNugteren Could you please have a look at the results? It only produced ":". Is this correct? I used openBLAS as the comparison library and the exe did not show any test cases like @0cc4m.
A770 test.txt

[tangjinchuan]

@0cc4m I remembered one test update from 1.6.1 to 1.6.2 is here afb3d8a
If you were using 1.6.2 previously with newly tunned results, I guess this worths trying: Replacing the xgemm folder of 1.6.1 by 1.6.2 so that 1.6.1 could have the xgemm tuning results. 6e2ab6e#diff-aad42a9516d39f8f4e703348e73e0de9a15f068041676d7a3d32510864abb9d9
Then try testing 1.6.1 to see if it is the problem related to updated preprocessor.
Sorry for I could not do it for it's getting late today here in my place.

[CNugteren]

@CNugteren Could you please have a look at the results? It only produced ":". Is this correct? I used openBLAS as the comparison library and the exe did not show any test cases like @0cc4m. A770 test.txt

No that is not testing anything. I think it just crashes silently? The test results should look like output_1.6.1.txt in the first message in this thread.

BTW, it could also be that there is a bug in CLBlast or in the tuner. If @0cc4m has time, perhaps he/she could change some values in the tuning database, re-compile, and re-run the tests. E.g. starting from the GEMM values themselves in https://github.com/CNugteren/CLBlast/pull/503/files#diff-aad42a9516d39f8f4e703348e73e0de9a15f068041676d7a3d32510864abb9d9R164. If we know which line in that PR causes the issues, then we are one step further. Then, next we could play with individual values of the tuning parameters, and try to find out which combinations of parameters are invalid, and which ones are still valid. But that might be some work.

[tangjinchuan]

The Xe laptop used to compile the test cases also only produce one ":". I guess it is not a single problem for A770 only.