Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PERF: faster access to the dtype for masked numeric arrays #52998

Merged
merged 2 commits into from
Apr 30, 2023
Merged

PERF: faster access to the dtype for masked numeric arrays #52998

merged 2 commits into from
Apr 30, 2023

Conversation

topper-123
Copy link
Contributor

@topper-123 topper-123 commented Apr 29, 2023 •
edited 

>>> import pandas as pd
>>> arr = pd.array(np.arange(100_000), dtype="Int32")
>>> %%timeit
... arr.dtype
... del arr._cache["dtype"]
2.09 µs ± 14.2 ns per loop  # main
234 ns ± 0.237 ns per loop  # this PR

Discovered working on some performance issues with #52836.

@jbrockmendel
Copy link
Member

nice!

@phofl phofl added Performance Memory or execution speed performance NA - MaskedArrays Related to pd.NA and nullable extension arrays labels Apr 30, 2023
@phofl phofl added this to the 2.1 milestone Apr 30, 2023
@phofl phofl merged commit 37e9e06 into pandas-dev:main Apr 30, 2023
33 checks passed
@phofl
Copy link
Member

phofl commented Apr 30, 2023

thx @topper-123

@topper-123 topper-123 deleted the perf_masked_dtypes branch April 30, 2023 12:28
NumanIjaz pushed a commit to NumanIjaz/pandas that referenced this pull request May 1, 2023
@topper-123 @NumanIjaz
PERF: faster access to the dtype for masked numeric arrays (pandas-de…
847a428 
…v#52998)
@topper-123 topper-123 mentioned this pull request May 2, 2023
Merged
1 task
topper-123 added a commit to topper-123/pandas that referenced this pull request May 7, 2023
@topper-123
PERF: faster access to the dtype for masked numeric arrays (pandas-de…
54863eb 
…v#52998)
Rylie-W pushed a commit to Rylie-W/pandas that referenced this pull request May 19, 2023
@topper-123
PERF: faster access to the dtype for masked numeric arrays (pandas-de…
96b6a6f 
…v#52998)
Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
@topper-123 @im-vinicius
PERF: faster access to the dtype for masked numeric arrays (pandas-de…
caada0d 
…v#52998)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@phofl phofl phofl approved these changes

Assignees
No one assigned
Labels
NA - MaskedArrays Related to pd.NA and nullable extension arrays Performance Memory or execution speed performance
Projects
None yet
Milestone
2.1
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@topper-123 @jbrockmendel @phofl