Skip to content

PERF: ArrowExtensionArray.isna when zero nulls or all nulls #51630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Feb 25, 2023
Merged

PERF: ArrowExtensionArray.isna when zero nulls or all nulls #51630

merged 2 commits into from
Feb 25, 2023

Conversation

lukemanley
Copy link
Member

Perf improvement in ArrowExtensionArray.isna when array has zero nulls or is all nulls.

ASV added for DataFrame.isna:

       before           after         ratio
     [f9438649]       [3cb3cf47]
     <main>           <perf-arrow-isna>
-      32.1±0.5ms      7.95±0.08ms     0.25  frame_methods.Isna.time_isna('float64[pyarrow]')

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

@lukemanley lukemanley added Performance Memory or execution speed performance Arrow pyarrow functionality labels Feb 25, 2023
@phofl phofl added this to the 2.1 milestone Feb 25, 2023
@phofl phofl merged commit e057427 into pandas-dev:main Feb 25, 2023
@phofl
Copy link
Member

phofl commented Feb 25, 2023

thx @lukemanley

@lukemanley lukemanley deleted the perf-arrow-isna branch March 17, 2023 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@phofl phofl phofl approved these changes

Assignees
No one assigned
Labels
Arrow pyarrow functionality Performance Memory or execution speed performance
Projects
None yet
Milestone
2.1
Development

Successfully merging this pull request may close these issues.
2 participants
@lukemanley @phofl