TO DO til 11/11 sujin

[5ujinKang]

What aclnn_ops does (only for our project)

llama.cpp/ggml/src/ggml-cann/aclnn_ops.cpp

[5ujinKang]

structured state space model convolution
Now, we only need to touch aclnn_ops.cpp : ssm_conv

HOW TO IMPLEMENT

1. Depthwise Conv1D (most exact)
2. Use im2col + MatMul
3. BatchMatMul

We need to wrap aclnn function to use in cann
search with add and explain to Aleksei

https://github.com/0Marble/llama.cpp/blob/sujin/ggml/src/ggml-cann/aclnn_ops.cpp

aclnn functions

1. Depthwise Conv 2d (i think we can set 1 to make it as 1D)
   https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/aolapi/context/aclnnConvDepthwise2d.md

2. Im2Col (aclnnIm2col)
   https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/aolapi/context/aclnnIm2col.md

3. GEMM (aclnnGemm)
   https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/aolapi/context/aclnnGemm.md

[5ujinKang]

CPU ver
`// ggml_compute_forward_ssm_conv

static void ggml_compute_forward_ssm_conv_f32(
const ggml_compute_params * params,
ggml_tensor * dst) {
const ggml_tensor * src0 = dst->src[0]; // conv_x
const ggml_tensor * src1 = dst->src[1]; // conv1d.weight

const int ith = params->ith;
const int nth = params->nth;

const int nc  = src1->ne[0]; // d_conv
const int ncs = src0->ne[0]; // d_conv - 1 + n_t
const int nr  = src0->ne[1]; // d_inner
const int n_t =  dst->ne[1]; // tokens per sequence
const int n_s =  dst->ne[2]; // number of sequences in the batch

GGML_ASSERT( dst->ne[0] == nr);
GGML_ASSERT(src0->nb[0] == sizeof(float));
GGML_ASSERT(src1->nb[0] == sizeof(float));
GGML_ASSERT(src0->nb[1] == src0->ne[0]*sizeof(float));

// rows per thread
const int dr = (nr + nth - 1)/nth;

// row range for this thread
const int ir0 = dr*ith;
const int ir1 = MIN(ir0 + dr, nr);
const int ir  = ir1 - ir0;

for (int i3 = 0; i3 < n_s; ++i3) {
    for (int i2 = 0; i2 < n_t; ++i2) {
        // {d_conv - 1 + n_t, d_inner, n_seqs}
        // sliding window
        const float * s = (const float *) ((const char *) src0->data + ir0*(src0->nb[1]) + i2*(src0->nb[0]) + i3*(src0->nb[2])); // {d_conv, d_inner, n_s}
        const float * c = (const float *) ((const char *) src1->data + ir0*(src1->nb[1])); // {d_conv, d_inner}
        float * x = (float *) ((char *) dst->data + ir0*(dst->nb[0]) + i2*(dst->nb[1]) + i3*(dst->nb[2])); // {d_inner, n_t, n_s}

        // TODO: transpose the output for smaller strides for big batches?
        // d_inner
        for (int i1 = 0; i1 < ir; ++i1) {
            // rowwise dot product
            // NOTE: not using ggml_vec_dot_f32, because its sum is in double precision
            float sumf = 0.0f;

            // d_conv
            for (int i0 = 0; i0 < nc; ++i0) {
                sumf += s[i0 + i1*ncs] * c[i0 + i1*nc];
            }
            x[i1] = sumf;
        }
    }
}

}
`

[5ujinKang]

algorithm

For each batch index i3 in [0 .. n_s):
For each token index i2 in [0 .. n_t):
For each channel (row) i1 in [0 .. d_inner):

    sum = 0

    For each kernel position k in [0 .. d_conv):
        sum += conv_x[t = i2 + k, channel = i1, batch = i3]
               * weight[k, channel = i1]

    output[channel = i1, token = i2, batch = i3] = sum