NVIDIA and CUDA

NVIDIA and CUDA

MLPL provides CUDA execution on Linux x86-64 with NVIDIA hardware. It uses Hugging Face Candle and cudarc, exposes device("cuda"), supports GPU forward operations, and accelerates recognized LoRA and MLP training shapes.

Platform and build gate

CUDA crates are gated on all three conditions:

• Cargo feature cuda
• target_os = "linux"
• target_arch = "x86_64"

The tested reference configuration uses an RTX 5060 Ti with CUDA 13.2 and compute capability sm_120. Other NVIDIA/toolkit combinations need their own compatible CUDA_COMPUTE_CAP, driver, and Candle/cudarc support.

Representative environment and test command from the CUDA training component:

export PATH="/opt/cuda/bin:$PATH"
export CUDA_ROOT=/opt/cuda
export CUDA_PATH=/opt/cuda
export CUDA_COMPUTE_CAP=120
export LD_LIBRARY_PATH="/opt/cuda/lib64:$LD_LIBRARY_PATH"

cd sw-mlpl/components/cuda-rt
cargo test -p mlpl-cuda-train --features cuda -- --test-threads=1

Adjust the paths and compute capability to the host. For a connected UI, build/run mlpl-serve with the CUDA feature from its owning workspace and serve the web assets.

CUDA flow

flowchart LR
    Client[Browser or terminal client] --> Serve[CUDA-enabled mlpl-serve]
    Serve --> Eval[MLPL evaluator]
    Eval --> Scope[device cuda scope]
    Scope --> Ops[CUDA dispatch]
    Ops --> Candle[Candle tensors and autograd]
    Candle --> GPU[NVIDIA GPU]
    GPU --> Metrics[Loss and device telemetry]
    Metrics --> Client

Loading

Capabilities

• CUDA runtime crates for conversion, matmul, reshape, transpose, arithmetic, unary activations, reductions, normalization, and cross-entropy-related operations.
• Traceable CUDA forward primitives for embedding, RMS normalization, causal attention, LoRA linear, and demo-model assembly.
• Candle-based autodiff and Adam update in supported fast paths.
• CPU/CUDA fp32 parity tests on real GPU hardware.
• CUDA LoRA fine-tune and CUDA tic-tac-toe fine-tune demos.
• mlpl-serve --features cuda reports cuda through /v1/devices, allowing the connected web UI to enable CUDA demos and disable unavailable MLX demos.

Critical limitation

CUDA does not provide general arbitrary-model GPU training through a device-aware autograd tape. The evaluator recognizes specific backend training forms, notably the head-only LoRA path and the two-linear LoRA MLP. Unsupported training graphs use CPU tape execution. Ordinary supported forward operations may still dispatch to CUDA, but that does not make the whole training loop GPU-resident.

CUDA runs inside a CUDA-enabled mlpl-serve on the GPU host. A separate mlpl-cuda-serve peer, multi-GPU execution, NCCL, distributed training, and LAN auto-discovery are not supported.

Best current uses

• Run and study the CUDA LoRA and tic-tac-toe demonstrations.
• Validate CPU/CUDA numerical parity.
• Connect the full browser UI to a Linux/NVIDIA compute host.

Source references: components/cuda-rt, components/cuda-model, components/cuda-eval, and the CUDA training crate notes.