Feature Request: Enable split-mode row/tensor for kimi-k3-fullsize-vision branch

[BenWongCityuCS]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Based on the unsloth fork (unslothai#48) which builds on PR #26185 and adds multimodal support for Kimi K3, I would like to request -sm row and, if possible, -sm tensor (splitting weights and KV cache) for multi-GPU inference.

Currently, for Kimi K3, only -sm layer is functional. My current command:

llama-server \
  -m Kimi-K3-UD-Q2_K_XL-00001-of-00019.gguf \
  --mmproj mmproj-BF16.gguf \
  -c 1048576 -ngl all -fa on -b 8192 -ub 2048 \
  --predict 65536 -np 4 \
  -sm layer -mg 0 \
  --port 31001 --host 0.0.0.0

Hardware: 8× NVIDIA H20 141GB, CUDA 12.8.

---

Motivation

With -sm layer on this setup, decode throughput for Kimi K3 is only about 13 tokens/s, which is too slow. I would like -sm row (or -sm tensor) so that weights and KV cache can be split by rows across GPUs, allowing parallel computation per layer and improving decode speed.

Possible Implementation

No response