Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use engine flox if array is ordered? #262

Closed
mathause opened this issue Sep 6, 2023 · 2 comments · Fixed by #266
Closed

use engine flox if array is ordered? #262

mathause opened this issue Sep 6, 2023 · 2 comments · Fixed by #266
Labels
enhancement New feature or request performance

Comments

@mathause
Copy link
Contributor

mathause commented Sep 6, 2023

I find it a bit unfortunate that a quite common case (ordered groups) is 5-15 times slower with flox installed than without (pydata/xarray#7730). One option is to prefer engine="flox" if group_idx is ordered.

Is that something that could be considered? Or would that cause problem for the duck array support as discussed in engines?

I thought about opening an issue in numpy-groupies to add reduceat but maybe it's more relevant here.

@dcherian
Copy link
Collaborator

dcherian commented Sep 7, 2023

Yes we could use engine=None by default and check for sortedness of group_idx here:

group_idx = group_idx.reshape(-1)

I use this check already:

if (len(uniques) == size) and (uniques == np.arange(size, like=array)).all():

This would be a nice PR!

@dcherian dcherian added the enhancement New feature or request label Sep 7, 2023
@dcherian
Copy link
Collaborator

dcherian commented Sep 7, 2023

Sorry I meant

issorted = (group_idx[:-1] <= group_idx[1:]).all()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants