Skip to content

vt::Conv2d has no CUDA provider, so the shared 2-D convolution seam cannot run on a GPU #2709

Description

@localai-org-maint-bot

Row: KERNEL-VT-CONV2D-CUDA

vt::Conv2d (include/vt/ops.h:4428) has exactly ONE provider in the tree:

$ grep -rn "OpId::kConv2d" src/ include/
src/vt/op_provider.cpp:566:    case OpId::kConv2d:
src/vt/ops.cpp:3282:  reinterpret_cast<Conv2dFn>(GetOp(OpId::kConv2d, q.device.type))(q, out, x, weight, bias, args);
src/vt/cpu/cpu_conv2d.cpp:111:    RegisterOp(OpId::kConv2d, DeviceType::kCPU,

src/vt/cpu/cpu_conv2d.cpp:111 is the CPU registration and there is no CUDA
one, so vt::Conv2d on a CUDA queue resolves no op. Any model that routes a
2-D convolution through the shared seam therefore ships a capability that
cannot run on the only device class that could serve it.

Why this is being filed now

dots3-note W7a (#2703) ports the dots speech encoder, whose stem is three
Conv2d(kernel=3, stride=2) layers (nvidia/audio_encoder.py:466-474 @
9035151d6). It composes them as host im2col + vt::MatmulBT and records ONE
EXACT TRACKED EXCEPTION to the kConv2d seam, with this issue as the reason.
The in-tree precedent is include/vllm/model_executor/models/whisper_audio.h:33,
which composes its Conv1d stem the same way and says "no new CUDA kernel".

The exception is the right call for that slice — a stem routed through
kConv2d would fault on the queue every real deployment uses — but the gap
outlives it. It is a vt gap, not a model gap.

What closing it looks like

A CUDA provider for OpId::kConv2d matching the CPU one's contract
(src/vt/cpu/cpu_conv2d.cpp), gated the way the other CUDA op ports are: a
red-first CPU/CUDA agreement test over the shapes the registered callers use,
plus the arch gate check-cuda-op-arch-gate already applies. Then each im2col
composition that exists only because of this gap can be reconsidered on a
measurement rather than on availability.

Not in scope here

vt::Conv3d (ops.h:4475) and vt::Conv1d (ops.h:4609) have their own
provider tables and are not measured by this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions