Ggml/cpu col2im 1d

[ServeurpersoCom]

Overview

cpu: add GGML_OP_COL2IM_1D

CPU part of #23424, split per review feedback; the CUDA backend follows in a separate PR.

Modern neural audio vocoders (the BigVGAN family and its descendants) build their generator from upsampling blocks: a transposed 1D convolution followed by an AMP / Snake stack. The transposed conv is the upsampler, Snake ( #22667 ) is the periodic activation, and both sit on the hot path of every generated frame.

A ConvTranspose1d factorizes exactly as a GEMM followed by an overlap-add:

    columns = mul_mat(weight[IC, K*OC], input[IC, T_in])  -> [K*OC, T_in]
    signal  = col2im_1d(columns)                          -> [T_out, OC]
    with T_out = (T_in - 1)*s0 + K - 2*p0

Keeping the channel contraction inside ggml_mul_mat lets it ride the tuned (and quantizable) matmul kernels and tensor cores, leaving col2im_1d as a thin, memory bound overlap-add: each output reads only ceil(K/s0) columns.

The existing ggml_conv_transpose_1d takes the naive route: a single direct kernel that folds the IC contraction into the scatter and rescans the full input per output element. For a vocoder generator running this op many times per second over long sequences, that is the bottleneck. The GEMM + col2im split removes it and unlocks F16 / BF16 / quantized weights.

This is the upsampling primitive used across three downstream GGML projects: acestep.cpp (music generation), omnivoice.cpp (multilingual TTS) and qwentts.cpp (Qwen3-TTS 12Hz DAC decoder). It pairs with GGML_OP_SNAKE (the other half of the AMP block), and as with Snake the implementation has been exercised and validated on all backends by others before upstreaming.

Additional information

GGML_OP_COL2IM_1D for CPU, F32 / F16 / BF16 with an F32 accumulator, parallelized over output channels.

Backend coverage is already in place in test-backend-ops: eight geometries across all three types, including the canonical kernel = 2*stride upsampling shape, kernel < stride (gap positions are zeroed), kernel not a multiple of stride, both cropping variants and the single column edge case, plus three perf entries at real vocoder stage shapes reporting memory bandwidth. The CUDA follow-up will be validated against this grid with zero additional test code.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES Opus / MCP rootless container with Nvidia GPU

[ServeurpersoCom]

CPU Benchmark with 3 real-world use cases :

col2im-cpu-test.cpp.txt

root@pod:/mnt/workspace/col2im-bench# ./col2im-cpu-test
col2im-cpu-test: threads = 32, iters = 5
full ConvTranspose chain per decode, F32 weights, min over iters, all paths fed their preferred layout
MUL_MAT+COL2IM_1D: this PR, ConvTranspose1d as GEMM + overlap-add
MUL_MAT+IM2COL_BACK: existing GGML trick, strictly equivalent mathematically, but ugly (backward op, dummy kernel tensor, F32 CPU only)
CONV_TRANSPOSE_1D: the naive op traditionally used by every project unaware it can be accelerated

acestep.cpp T0 = 1024 (81.9s)

MUL_MAT+COL2IM_1D = 689.083 ms, Cosine Similarity = 0.999999999998
MUL_MAT+IM2COL_BACK = 743.477 ms (1.08x), Cosine Similarity = 0.999999999998
CONV_TRANSPOSE_1D = 2413.170 ms (3.50x), Cosine Similarity reference

qwentts.cpp T0 = 1500 (30.0s)

MUL_MAT+COL2IM_1D = 200.650 ms, Cosine Similarity = 0.999999999998
MUL_MAT+IM2COL_BACK = 273.453 ms (1.36x), Cosine Similarity = 0.999999999998
CONV_TRANSPOSE_1D = 582.309 ms (2.90x), Cosine Similarity reference

omnivoice.cpp T0 = 750 (30.0s)

MUL_MAT+COL2IM_1D = 67.129 ms, Cosine Similarity = 0.999999999999
MUL_MAT+IM2COL_BACK = 93.729 ms (1.40x), Cosine Similarity = 0.999999999999
CONV_TRANSPOSE_1D = 295.380 ms (4.40x), Cosine Similarity reference

[ggerganov]

Need to fix the RPC: https://github.com/ggml-org/llama.cpp/actions/runs/27053569438/job/79853460419?pr=24206#step:7:126

[ggerganov]

I think you just need to bump the RPC protocol patch version.

[ServeurpersoCom]

Yes, I remember doing that back when Snake was a separate operator and not a graph fusion pattern. I had launched the entire CI locally, which I forgot to do on this PR. I'll fix that quickly.