wrong output of aie simulation of int8 mm_kernel0

[hongzhengTian]

Dear Author,

I am currently practicing small-scale matrix multiplication using a single AIE tile. I have tried mm_kernel0.cc for various data types in the src path of your project (as mm_kernel0 includes two inputs and one output, theoretically it should be the basic unit capable of performing matrix multiplication).

I created separate projects for each data type of mm_kernel0.cc to conduct AIE Simulation. Moreover, I generated the test data and Golden data according to the required data type and size by using Python code.

According to my AIE simulator results, the simulation data for int16, int32, and fp32 data types of mm_kernel0.cc are all correct. However, the int8 data type alone does not yield the correct simulation results, which I find puzzling. For convenience, I have attached my test project for the int8 type. In this project, the input matrices used for testing are two identical 01 diagonal matrices. I transposed them for AIE input (as you know, AIE matrix multiplication requires column-wise storage of matrices). You can directly see the two transposed input matrices, along with the golden result, and the AIE output I saved, in the data/ directory. You'll notice that these two results are completely different. If you have time, you can recompile the simulation to get my results by running python int8_test_data_gen.py && make all. You can also check the other input matrix if modifying the Python code.

I don't think there is any issue with my testing process, as I can obtain correct output results for the other three data types using the same process, with only the int8 output being incorrect. I would like to know if there are any additional conditions to be aware of to obtain the correct results for int8 type mm_kernel0, or if there is something missing or incorrect in my process?

Thank you for your guidance.

mm_int8.zip

[JinmingZhuang]

In order to improve the efficiency of the single kernel under int8, we change the layout of all three matrices. The layout should be A: [H1/8][W1][8], B[W1/16][W2][16], C[H1/8][W2][8] ( Mat[0][1][2] ), where dim=2 is consecutive.

Hope this will help!

[hongzhengTian]

Thank you so much for your help! Now it works perfect!