New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: Series.dropna with non-nan dtype blocks #11159

Merged
merged 1 commit into from Oct 18, 2015

Conversation

Projects
None yet
4 participants
@sinhrks
Member

sinhrks commented Sep 21, 2015

Minor improvement when dropna actually does nothing. If OK for 0.17, I'll add a release note.

import numpy as np
import pandas as pd

s = pd.Series(np.random.randint(0, 1000, 50000000))

# before
%timeit s.dropna()
#1 loops, best of 3: 1.58 s per loop

# after
%timeit s.dropna()
#1 loops, best of 3: 568 ms per loop

@sinhrks sinhrks added this to the 0.17.1 milestone Sep 21, 2015

@jorisvandenbossche

View changes

Show outdated Hide outdated asv_bench/benchmarks/series_methods.py Outdated
@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 5, 2015

Contributor

sure, pls add a note in 0.17.1 whatsnew

Contributor

jreback commented Oct 5, 2015

sure, pls add a note in 0.17.1 whatsnew

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 9, 2015

Contributor

@sinhrks looks good, can you rebase / add a release note

Contributor

jreback commented Oct 9, 2015

@sinhrks looks good, can you rebase / add a release note

@sinhrks

This comment has been minimized.

Show comment
Hide comment
@sinhrks

sinhrks Oct 15, 2015

Member

Rebased, and added release note. Followings are asv results added in this PR.

    before     after       ratio
  [a89b96d4] [b2cb56a8]
    10.76ms    10.60ms      0.99  series_methods.series_dropna_datetime.time_series_dropna_datetime
-   12.07ms   726.61μs      0.06  series_methods.series_dropna_int64.time_series_dropna_int64
Member

sinhrks commented Oct 15, 2015

Rebased, and added release note. Followings are asv results added in this PR.

    before     after       ratio
  [a89b96d4] [b2cb56a8]
    10.76ms    10.60ms      0.99  series_methods.series_dropna_datetime.time_series_dropna_datetime
-   12.07ms   726.61μs      0.06  series_methods.series_dropna_int64.time_series_dropna_int64

jreback added a commit that referenced this pull request Oct 18, 2015

Merge pull request #11159 from sinhrks/dropna_perf
PERF: Series.dropna with non-nan dtype blocks

@jreback jreback merged commit 53a0db1 into pandas-dev:master Oct 18, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 18, 2015

Contributor

thanks!

Contributor

jreback commented Oct 18, 2015

thanks!

@sinhrks sinhrks deleted the sinhrks:dropna_perf branch Nov 7, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment