qblock

Q4S-32 quantized GEMV: encode/decode on CPU, CUDA kernels on GPU.

Build / test (CPU)

make test          # CPU tests

CUDA

make cuda_build       # compile ./validate and ./benchmark (needs nvcc; no GPU)
make cuda_validate    # build + run ./validate (needs a GPU)

# Timing (needs a GPU): append raw samples to a CSV
./benchmark --kernel copy --out results/runs.csv
./benchmark --kernel q4 --M 4096 --K 4096 --out results/runs.csv
# or: make cuda_bench
python3 tools/analyze_benchmark.py results/runs.csv

Override GPU arch if needed: make cuda_build NVCC_ARCH=native (or sm_75, etc.). Default is sm_86 (Ampere — RTX 30xx, A40, A5000, and newer).

Docker

Image has nvcc + gcc 13. make cuda_build works without a GPU (compile check only). make cuda_validate needs a host with an NVIDIA GPU (sm_86+ for the default arch, or rebuild with NVCC_ARCH=sm_75 for T4 / RTX 20xx) and the NVIDIA Container Toolkit.

# Build the toolchain image
docker compose build

# Interactive shell with repo bind-mounted
docker compose run --rm dev
# inside: make test && make cuda_build

# Run kernels (NVIDIA GPU + toolkit required)
docker compose run --rm gpu
# inside: make cuda_validate