v0 roadmap: metal runtime → comm benchmarks + kernels

[bniladridas]

We now have the initial gpucomm/core scaffold: a minimal SwiftPM Metal runtime (MetalContext + kernel loader), a CLI (gpucomm), and two first experiments (bandwidth + reduction).

Goal

Make this repo a low-level GPU compute runtime + benchmark lab for Apple Silicon that focuses on data movement, synchronization, and memory behavior (not abstractions).

Proposed next milestones

1. Kernel suite v0.1
   • Scan (prefix sum), matmul, and a proper memcpy-style bandwidth kernel
   • Parameterized kernels (problem size, threadgroup size) + correctness checks
2. Benchmark harness v0.2
   • Standard runner: warmup, repetitions, percentile stats
   • Output formats: human + --json
   • Stable timing: GPU timestamps + optional CPU wall time
3. CPU ↔ GPU transfer + residency v0.3
   • Upload/download latency + throughput (shared vs private + blit strategies)
   • Pinned/contiguous allocation experiments where applicable
4. Synchronization + threadgroup experiments v0.4
   • Barrier patterns, bank conflict probes, reduction/scan variants
   • Threadgroup scaling sweeps and occupancy-ish heuristics
5. CI + docs v0.5
   • GitHub Actions on macOS: swift build -c release
   • README: benchmark methodology + “how to interpret numbers”

Decisions needed

• Do we prioritize transfer benchmarks (CPU↔GPU) before adding more kernels?
• Should the CLI be strict subcommands only, or do we also support a config-driven batch runner (YAML/JSON)?
• What’s the canonical set of benchmark outputs we want to standardize across gpucomm repos?

Acceptance criteria for next PR

• Add 1 new benchmark end-to-end (kernel + runner + CLI) with correctness validation
• Add --json output for that benchmark
• CI stays green (swift build -c release)

[bniladridas]

Shipped transfer benchmark (CPU↔GPU).

• New: gpucomm bench transfer ... (supports h2d|d2h, shared+memcpy and private+blit, --json).
• Repro:
  • swift build -c release
  • ./.build/release/gpucomm bench transfer --size-kib 4 --iters 10000 --warmup 100 --direction h2d --mode private --strategy blit
  • ./.build/release/gpucomm bench transfer --size-kib 4 --iters 10000 --warmup 100 --direction d2h --mode private --strategy blit --json

Next: add a threadgroup-heavy primitive (scan) with correctness + benchmark output.

[bniladridas]

Shipped scan benchmark (single-threadgroup exclusive prefix sum, up to 1024 elements).

• New: gpucomm bench scan --n N --iters N --warmup N [--json]
• Repro:
  • swift build -c release
  • ./.build/release/gpucomm bench scan --n 1024 --iters 200 --warmup 20
  • ./.build/release/gpucomm bench scan --n 1000 --iters 200 --warmup 20 --json
• Correctness: compares GPU output vs CPU exclusive scan (ok=true|false).

Next: expand scan beyond 1024 (multi-dispatch), then add matmul.

[bniladridas]

Extended scan to handle sizes beyond 1024 (up to 1,048,576) via multi-threadgroup block-scan + block-sum scan + offset add.

• Repro:
  • swift build -c release
  • ./.build/release/gpucomm bench scan --n 65536 --iters 50 --warmup 10
  • ./.build/release/gpucomm bench scan --n 1048576 --iters 10 --warmup 2 --json
• Correctness: still validates GPU output vs CPU (ok=true|false).

Next: start matmul benchmark (tiling + threadgroup memory experiments).

[bniladridas]

Shipped matmul benchmark (FP32 GEMM) with two variants:

• naive (one thread = one C element)

• tiled16 (16x16 tiling using threadgroup memory)

• Repro:

  • swift build -c release
  • ./.build/release/gpucomm bench matmul --m 64 --n 64 --k 64 --iters 20 --warmup 5 --variant naive
  • ./.build/release/gpucomm bench matmul --m 256 --n 256 --k 256 --iters 50 --warmup 10 --variant tiled16 --json

• Output includes gflops and a small-size correctness check (ok=true|false).

Next: add matmul tuning knobs (tile size / vectorization) + threadgroup scaling sweeps.

[bniladridas]

Extended matmul with threadgroup tuning for the naive variant and added a sweep command.

• New flags (naive only): --tg-x, --tg-y
• New: gpucomm bench matmul-sweep (prints a few naive TG shapes + tiled16 for comparison; supports --json).

Repro:

• swift build -c release
• ./.build/release/gpucomm bench matmul --m 64 --n 64 --k 64 --iters 10 --warmup 2 --variant naive --tg-x 16 --tg-y 8
• ./.build/release/gpucomm bench matmul-sweep --m 256 --n 256 --k 256 --iters 10 --warmup 2 --json

Next: parameterize tiling variants (e.g., 8/16/32) and add a consistent sweep harness shared across benches.

[bniladridas]

Extended matmul tiled variants: added tiled8 and tiled32 (in addition to tiled16).

• New: gpucomm bench matmul --variant tiled8|tiled16|tiled32
• matmul-sweep now prints tiled8/16/32 for comparison.

Repro:

• swift build -c release
• ./.build/release/gpucomm bench matmul --m 64 --n 64 --k 64 --iters 10 --warmup 2 --variant tiled8
• ./.build/release/gpucomm bench matmul --m 128 --n 128 --k 128 --iters 10 --warmup 2 --variant tiled32
• ./.build/release/gpucomm bench matmul-sweep --m 256 --n 256 --k 256 --iters 5 --warmup 1

Next: factor sweep logic into a shared harness and add CSV/JSONL output modes across benches.

[bniladridas]

Included structured output formats across benches.

• New flag: --format
  • bandwidth|scan|matmul|transfer: human|json|csv
  • matmul-sweep: human|jsonl|csv
• --json still works as a compatibility alias (json for single-result benches, jsonl for sweeps).

Repro:

• ./.build/release/gpucomm bench bandwidth --size-mib 16 --iters 50 --mode shared --format json
• ./.build/release/gpucomm bench transfer --size-kib 4 --iters 1000 --warmup 50 --direction d2h --mode private --strategy blit --format csv
• ./.build/release/gpucomm bench matmul-sweep --m 128 --n 128 --k 128 --iters 2 --warmup 1 --format csv

Next: factor output formatting into a shared helper (so other sweeps/benches can reuse the same CSV/JSONL plumbing).

[bniladridas]

Included scan-sweep and refactored CLI parsing/output helpers for reuse.

• New: gpucomm bench scan-sweep --ns CSV --iters N --warmup N --format human|jsonl|csv
• Refactor: extracted ArgReader + output helpers into Sources/gpucomm/Args.swift + Sources/gpucomm/Output.swift.

Repro:

• swift build -c release
• ./.build/release/gpucomm bench scan-sweep --ns 1024,4096,65536,1048576 --iters 50 --warmup 10 --format jsonl

Next: reuse the same sweep harness for transfer and bandwidth (size sweeps) + add a shared stats runner.

[bniladridas]

Included bandwidth-sweep (size sweep) with JSONL/CSV output.

• New: gpucomm bench bandwidth-sweep --sizes-mib CSV --iters N --mode shared|private --format human|jsonl|csv

Repro:

• swift build -c release
• ./.build/release/gpucomm bench bandwidth-sweep --sizes-mib 1,4,16,64 --iters 200 --mode private --format jsonl

Next: transfer-sweep (sizes × direction × mode/strategy) and a shared stats runner (warmup/reps/percentiles).

[bniladridas]

Included transfer-sweep (sizes × direction × mode/strategy) with JSONL/CSV output.

• New: gpucomm bench transfer-sweep --sizes-kib CSV --iters N --warmup N --direction h2d|d2h|both --mode shared|private|both --format human|jsonl|csv
• Strategy is implicit per mode: shared→memcpy, private→blit.

Repro:

• swift build -c release
• ./.build/release/gpucomm bench transfer-sweep --sizes-kib 1,4,64 --iters 5000 --warmup 200 --direction both --mode both --format jsonl

Next: shared stats runner (warmup, reps, percentiles) so sweeps can report p50/p95 and reduce noise.

[bniladridas]

Included a shared stats runner for sweeps: --reps N + percentile summaries.

• New helper: Sources/gpucomm/Stats.swift (p50/p95 + min/max/mean).
• Sweeps now accept --reps N and output percentiles:
  • bandwidth-sweep: gibps_p50/p95, gpu_ms_p50/p95
  • scan-sweep: avg_us_p50/p95, gpu_ms_p50/p95
  • matmul-sweep: gflops_p50/p95, avg_us_p50/p95, gpu_ms_p50/p95
  • transfer-sweep: avg_us_p50/p95, bytes_per_second_p50/p95, plus gpu_ms_p50/p95 when applicable

Repro:

• ./.build/release/gpucomm bench transfer-sweep --sizes-kib 1,4 --iters 1000 --warmup 50 --reps 5 --direction both --mode both --format jsonl
• ./.build/release/gpucomm bench matmul-sweep --m 128 --n 128 --k 128 --iters 2 --warmup 1 --reps 5 --format csv

Next: expose the same --reps + percentiles for single-shot benches (non-sweep) and add p99 if needed.

[bniladridas]

Extended single-shot benches to support --reps N with percentile summaries (p50/p95).

• bandwidth: --reps outputs gibps_p50/p95 + gpu_ms_p50/p95
• scan: --reps outputs avg_us_p50/p95 + gpu_ms_p50/p95 + ok
• matmul: --reps outputs gflops_p50/p95 + avg_us_p50/p95 + gpu_ms_p50/p95 + ok
• transfer: --reps outputs avg_us_p50/p95 + bytes_per_second_p50/p95 (+ gpu_ms_p50/p95 when applicable)

Repro:

• ./.build/release/gpucomm bench bandwidth --size-mib 16 --iters 50 --mode private --reps 5 --format json
• ./.build/release/gpucomm bench transfer --size-kib 4 --iters 1000 --warmup 50 --reps 5 --direction d2h --mode private --strategy blit --format csv

Next: optionally add --p99 and a mode to emit raw per-rep samples (JSONL) for deeper analysis.

[bniladridas]

Included macOS CI to keep the SwiftPM build green.

• New workflow: .github/workflows/ci.yml
• Runs swift build -c release on macos-latest for pushes + PRs.

Next: (optional) add a "no-GPU" smoke run that only prints --help (doesn’t require a Metal device).

[bniladridas]

Improved CI robustness by making gpucomm --help run without requiring a Metal device, and added a CI smoke step.

• CLI: early-exit help path before creating MetalContext
• CI: runs ./.build/release/gpucomm --help after swift build -c release