CUDA Graph phase 3 - memcpy nodes

[rparolin]

Cover cuMemcpyAsync and cover cuMemsetAsync.

https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__MEM.html#group__CUDA__MEM_1g5f26aaf5582ade791e5688727a178d78

cuMemsetAsync has multiple bit-width APIs. We can wrap them nicely from the python side so the user doesn't need to worry about what bit-width variant they need to call.

https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__MEM.html#group__CUDA__MEM_1g7d805e610054392a4d11e8a8bf5eb35c

[Andy-Jost]

Is cuMempyAsync covered by Buffer.copy_to and Buffer.copy_from or is something special needed for graphs?

Could we similarly cover cuMemsetAsync by adding a method to Buffer?

[rparolin]

Is cuMempyAsync covered by Buffer.copy_to and Buffer.copy_from or is something special needed for graphs?
@leofang please advise.

Could we similarly cover cuMemsetAsync by adding a method to Buffer?
Yes, that's my understanding. We could have a call fill or clear ?

[Andy-Jost]

Ok, follow-up question:

We can wrap them nicely from the python side so the user doesn't need to worry about what bit-width variant they need to call

How do we envision this working? If someone says buf.fill(1) we still don't have information about the intended bit-width. Are we thinking something like buf.fill(value=1, width=4) ?

If the type is a specific NumPy dtype we can do it automatically. Is there a precedent for anything like that, or do we have a list of frameworks/dtypes that we support?

[Andy-Jost]

Please see the attached draft PR. Is that what we have in mind for Buffer.fill?