Add to_arrow_device() functions that accept views - #15465
Conversation
vyasr
left a comment
There was a problem hiding this comment.
Taking a peek while it's in draft (and rebased #15370 on these changes). Overall looks great here. I was originally hoping to unify the code paths a bit more, but the view path doesn't look all that verbose and it seems like it wouldn't improve readability much to unify. We could try implementing column equivalents of set_null_mask and set_view_to_buffer as class methods on the original to_arrow_device functor so that the operator specializations could be reused, but I don't really know how well that would work. WDYT?
| // TODO: Can't we just not set anything here? The arrow spec says that | ||
| // you can leave the pointer void if the size is 0. |
There was a problem hiding this comment.
Updated the comment. Let me know if I should just delete it.
| @@ -0,0 +1,231 @@ | |||
| /* | |||
There was a problem hiding this comment.
Moved from to_arrow_device.cu
| } | ||
| } | ||
| }; | ||
|
|
There was a problem hiding this comment.
Moved to to_arrow_schema.cpp
| cudf::column_view column; | ||
| rmm::cuda_stream_view stream; | ||
| rmm::mr::device_memory_resource* mr; |
There was a problem hiding this comment.
is there a particular reason to do this vs passing them as parameters to the functors?
There was a problem hiding this comment.
Just thought it was a bit cleaner since they are the same for every dispatch function.
Adding/removing parameters to every function got tedious.
zeroshade
left a comment
There was a problem hiding this comment.
Other than my nitpick and question, this seems good to me
vyasr
left a comment
There was a problem hiding this comment.
Some small feedback, but nothing worth blocking over except perhaps the release callback question. I worry that we're creating potential UB. I don't know if it's really specified explicitly anywhere in the spec, but throwing a C++ exception in a callback that may be called in C is definitely going to be implementation-defined behavior. Perhaps the best we can do is printing or logging the exception then returning safely? This feels like a hole in the spec that we should perhaps start a discussion about upstream.
|
/merge |
Description
Adds the following new interop functions
Also refactors some common code with the ownership transfer version of these APIs.
And moves the
to_arrow_schema()functions to a separate .cpp file.Checklist