test(gap9): add the LoRA and QLoRA CCT training models to CI

[runwangdl]

Adds the LoRA and QLoRA CCT training models to CI, and carries the kernel and
frontend work they depend on. This supersedes #50, #51 and #53, which are closed in
favour of this one.

Models

CCT_LoRA_R1 and CCT_QLORA_FT join the GAP9 tiled training configurations at
--l1 122000.

Gemm with a one-dimensional bias (was #50)

GEMMLayer.computeShapes widened the C operand to [M, N] unconditionally, because
the kernel walked the bias as a full matrix. A Linear's bias is O values shared by
every output row, so the widened operand stored the same vector M times: on CCT-2
at 64 tokens, 32 KB to carry 512 B. The kernel now takes a bias row stride, O
reproducing the old walk exactly and 0 re-reading one vector per row, so a single
kernel serves both layouts and the previous behaviour is bit-identical at stride O.

The tiling side matters as much as the kernel. serializeTilingSolution built the C
cube from the output cube, so a [O] bias was still transferred as if it were
[M, O]. A broadcast bias now gets a cube of its own rank. Measured on the generated
code for cct_train at --l1 128000 --defaultMemLevel L3, node_84:

	C_transfer_size	total
before	{28672, 4096}	32768 B
after	{448, 64}	512 B

512 B is the whole bias, O = 128 floats across the two O tiles of 112 and 16. The
oversized cube is invisible whenever the operand is already resident, which is why
the L2 job passed while the L3 and promote jobs each failed with 4 errors out of 4.

Broadcast Add (was #51)

The same treatment for Add, for the graphs whose Linear layers export as
MatMul + Add rather than Gemm.

Fused dequantisation (was #53) — present but NOT yet effective

The int8 kernels, bindings and the topology pass are here, but the fold does not
currently take effect and no claim is made for it. The pass lost its consumer-rewiring
loop, so it never pointed a MatMul/Gemm/Conv at the int8 tensor and never set
dequant_scale. Restoring the loop then trips hoistConstant, which asserts a
constant feeds at most one node: a folded weight is read by both the forward MatMul
and its backward Gemm, and this pass runs after the frontend duplication. Giving each
consumer its own int8 copy clears that one and exposes a second multi-consumer
constant from lowering.

An earlier revision of this description reported the fused kernels being called on
CCT-LoRA. That was not the fold working. It was the binding-order defect below
retyping an fp32 weight to int8, which is the same defect that failed the FP32 Conv
kernel test 512 of 512.

The 51.55M cycle figure measured earlier came from a tree that had the consumer loop
intact, so it is not reproduced by what is on this branch. Getting the fold working
needs the constant-sharing question settled — a ConstantBuffer is global and
read-only, so sharing across consumers is sound and is what keeps the memory win, but
that means relaxing the assertion rather than copying around it. That work belongs in
its own PR.

Binding order: fp32 before folded-int8

A binding is selected before a weight constant's type is fixed, and
typeInferGlobalCtxt types that constant from the SELECTED binding's input_types.
Listing the int8 variant first therefore claims ordinary fp32 weights and retypes
them to int8, silently quantising them:

Kernels/FP32/Conv/Regular_2D_NoBias   512 errors out of 512
Kernels/FP32/GEMM/Regular             implicit declaration of PULP_Gemm_fp32_i8_fp32_fp32
Models/CCT/FP32/CCT_2_32_32_128       10 errors out of 10

fp32 is matched first across GAP9FloatGEMMBindings, GAP9FloatConv2DBindings,
GAP9MatMulBindings, PULPFloatConv2DBindings and PULPMatMulBindings.

Constant duplication after folding

cct_lorar1_train failed to generate at all, on the model-adding commit by itself:

AssertionError: Constant node_0_classifier_blocks_0_self_attn_proj_Transpose__0_tensor
has more than one output

Folding collapses Constant -> Transpose -> Variable into one Constant, and a
training graph has the forward MatMul and the backward Gemm both reading it.
_duplicateConstants runs before _foldConstants, so nothing splits what the fold
merged. It now runs again afterwards; it only touches tensors with more than one
consumer, so it is a no-op when the fold introduced none.

🤖 Generated with Claude Code