Migrate interop functionality to separate module #15310
Labels
feature request
New feature or request
pylibcudf
Issues specific to the pylibcudf package
Python
Affects Python cuDF API.
Currently pylibcudf Table and Scalar support
[to|from]_arrow
methods. This is an antipattern in pylibcudf where essentially all functionality is provided via free functions operating on objects rather than via class methods. Furthermore, the current approach forces using pyarrow Cython in scalar.pyx and scalar.pxd, which in turn essentially means that every part of cudf uses arrow Cython. As #15193 progresses we will also be looking to isolate the pyarrow dependency entirely from pylibcudf and probably make it optional (cudf will still require it), so it would be best to put all such logic directly into the interop module. The exact implementations will change as we migrate to a capsule-based FFI rather than the current approach of passing typed arrow objects directly (which requires linking to arrow).While making this change, we should also update the scalar converter to not require column metadata. We should be able to spoof that since it is only really necessary to create the intermediate column and is discarded immediately afterwards.
The text was updated successfully, but these errors were encountered: