You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Incremental compile times feel like they have gotten slower some time ago. While investigating I notices that a large amount of llvm code is generated for the comparison kernels, especially dyn and dict versions. These end up calling the generic BooleanArray::from_iter method which therefore gets duplicated for each possible iterator type.
This seems to indicate that nearly half of the llvm code is caused by the comparison kernels.
Describe the solution you'd like
I'm not sure whether there is a good solution for this, but wanted to document these findings.
The kernels use macros very extensively, but replacing these with generic functions will probably lead to the same amount of llvm lines because of monomorphization. Maybe some of the non-generic common code could be extracted into helper methods.
Describe alternatives you've considered
The dyn kernels could also be put behind a feature flag, but that would again complicate the test matrix.
The text was updated successfully, but these errors were encountered:
It occurs to me that #2038 might actually help with this, as it reduces the amount of logic in the FromIterator implementation, and delegates it to the Builder
I'm closing as these comparison kernels are now gated with feature flags, and crates can also choose not to depend on arrow-ord at all. Feel free to re-open if there is additional work to be done in this area
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Incremental compile times feel like they have gotten slower some time ago. While investigating I notices that a large amount of llvm code is generated for the comparison kernels, especially dyn and dict versions. These end up calling the generic
BooleanArray::from_iter
method which therefore gets duplicated for each possible iterator type.Removing the dyn and dict comparison kernels gives the following number of lines (but this is of course not a realistic option):
This seems to indicate that nearly half of the llvm code is caused by the comparison kernels.
Describe the solution you'd like
I'm not sure whether there is a good solution for this, but wanted to document these findings.
The kernels use macros very extensively, but replacing these with generic functions will probably lead to the same amount of llvm lines because of monomorphization. Maybe some of the non-generic common code could be extracted into helper methods.
Describe alternatives you've considered
The
dyn
kernels could also be put behind a feature flag, but that would again complicate the test matrix.The text was updated successfully, but these errors were encountered: