chore: drop async from CUDA traits

[0ax1]

Summary

Removes async from CUDA Vortex traits.

Execution model

For a given execution context, all work (with the exception to host buf copies) is enqueued onto a single CUDA stream and executes in FIFO order. Kernel launches are synchronous fire-and-forget: they enqueue work and return immediately. The returned Canonical may reference device buffers with in-flight writes; correctness is guaranteed by CUDA stream ordering, not by any CPU-side synchronization.

The only operations that require waiting for GPU work to complete are device-to-host copies, where the CPU must block until data lands in host memory.

To insert an explicit sync point, use await_stream_callback, which completes when all preceding stream work has finished.

[joseph-isaacs]

looks sus

[0ax1]

Which part exactly? Any T is DeviceRepr.

[0ax1]

Turns out, memory that is already allocated can be registered as pinned with cuMemHostRegister_v2, such that the copy to device will be async. See: https://developer.download.nvidia.com/compute/DevZone/docs/html/C/doc/html/group__CUDA__MEM_gf0a9fe11544326dabd743b7aa6b54223.html.

[robert3005]

FWIW when I read the docs they strongly encourage to use their allocator as that's faster than cuMemHostRegister/cuMemHostUnregister but in places where we can't do that we should definitely fallback to that method

[0ax1]

FWIW when I read the docs they strongly encourage to use their allocator as that's faster than cuMemHostRegister/cuMemHostUnregister but in places where we can't do that we should definitely fallback to that method

That was my read too. Maybe we can think about this as a stop gap in the future before we wire in a "CUDA allocator".