-
Notifications
You must be signed in to change notification settings - Fork 713
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Encode Scalars as dyn Any
in Scalar dyn Kernels
#2842
Comments
I think this sounds like a great idea. I didn't realize Any was implemented for the basic types |
Hm, will this affect performance? |
I wouldn't expect it to have a material performance impact. It will perform a downcast of the scalar value once when it downcasts the array, and then proceed as normal |
One wrinkle the comes up here is that we need a way to encode the data type of the scalar, e.g. to allow adding both interval and duration scalars of varying types to temporal types such as dates and timestamps. We may need to introduce a Scalar trait 🤔 |
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
As expanded upon in #2837 currently the encoding of scalars in dyn kernels is inconsistent. The arithmetic kernels must be explicitly type hinted with the
ArrowPrimitiveType
, whilst the comparison kernels coerce the scalar type to the desired type.Taking a step back, users just want to give a kernel an array and a scalar of the correct type and have it work, with all the dispatch logic handled for it.
Describe the solution you'd like
I would like to suggest encoding scalars as
& dyn std::any::Any
, for example,Users can then write
This has the following advantages:
The only major downside is that users will need to be careful to correctly hint scalar types, i.e.
&1_i64
.Describe alternatives you've considered
We could not do this
Additional context
The text was updated successfully, but these errors were encountered: