/pandas

PERF: don't materialize arrays on checking in groupby #16413

Merged
merged 1 commit into from May 22, 2017

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
Groupby Performance
Projects
None yet
Milestone
  0.20.2
3 participants
@jreback @codecov @TomAugspurger
@jreback
Contributor

jreback commented May 22, 2017 • edited 

    before     after       ratio
  [a6fcec6c] [6edcdc0d]
-   15.41ms     9.48ms      0.62  groupby.groupby_size.time_groupby_size

this is more dramatic as the size increases as this should be pretty much a constant time operation (the actual size computation itself).

@jreback jreback added Groupby Performance labels May 22, 2017

jreback added this to the 0.20.2 milestone May 22, 2017

@jreback
Contributor

jreback commented May 22, 2017

cc @wesm @cpcloud
@jreback jreback PERF: don't materialize arrays on checking in groupby

All checks have passed

5 successful checks
ci/circleci — Your tests passed on CircleCI!
Details
@codecov
codecov/patch — Coverage not affected when comparing ee54722...400e14a
Details
@codecov
codecov/project — 90.42% (target 82%)
Details
continuous-integration/appveyor/pr — AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr — The Travis CI build passed
Details
400e14a
@codecov

codecov bot commented May 22, 2017

Codecov Report

Merging #16413 into master will increase coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16413      +/-   ##
==========================================
+ Coverage   90.41%   90.42%   +0.01%     
==========================================
  Files         161      161              
  Lines       51024    51023       -1     
==========================================
+ Hits        46133    46138       +5     
+ Misses       4891     4885       -6
Flag Coverage Δ
#multiple 88.26% <ø> (+0.01%) ⬆️
#single 40.17% <ø> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/base.py 95.74% <ø> (-0.01%) ⬇️
pandas/core/common.py 91.39% <0%> (+0.33%) ⬆️
pandas/io/formats/format.py 96.02% <0%> (+0.36%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee54722...400e14a. Read the comment docs.
@codecov

codecov bot commented May 22, 2017 • edited

Codecov Report

Merging #16413 into master will increase coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16413      +/-   ##
==========================================
+ Coverage   90.41%   90.42%   +0.01%     
==========================================
  Files         161      161              
  Lines       51024    51023       -1     
==========================================
+ Hits        46133    46138       +5     
+ Misses       4891     4885       -6
Flag Coverage Δ
#multiple 88.26% <ø> (+0.01%) ⬆️
#single 40.17% <ø> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/base.py 95.74% <ø> (-0.01%) ⬇️
pandas/core/common.py 91.39% <0%> (+0.33%) ⬆️
pandas/io/formats/format.py 96.02% <0%> (+0.36%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee54722...400e14a. Read the comment docs.

@jreback jreback merged commit d5a681b into pandas-dev:master May 22, 2017
5 checks passed

5 checks passed

ci/circleci Your tests passed on CircleCI!
Details
codecov/patch Coverage not affected when comparing ee54722...400e14a
Details
codecov/project 90.42% (target 82%)
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@pcluo pcluo added a commit to pcluo/pandas that referenced this pull request May 22, 2017

@jreback @pcluo jreback + pcluo PERF: don't materialize arrays on checking in groupby (#16413) c772726

@TomAugspurger TomAugspurger added a commit to TomAugspurger/pandas that referenced this pull request May 29, 2017

@jreback @TomAugspurger jreback + TomAugspurger PERF: don't materialize arrays on checking in groupby (#16413) 
(cherry picked from commit d5a681b)
 61503ea

@TomAugspurger TomAugspurger added Backported and removed Needs Backport labels May 30, 2017

@TomAugspurger TomAugspurger added a commit that referenced this pull request May 30, 2017

@jreback @TomAugspurger jreback + TomAugspurger PERF: don't materialize arrays on checking in groupby (#16413) 
(cherry picked from commit d5a681b)
 af31450

This was referenced Jun 4, 2017

Closed
Open

@stangirala stangirala added a commit to stangirala/pandas that referenced this pull request Jun 11, 2017

@jreback @stangirala jreback + stangirala PERF: don't materialize arrays on checking in groupby (#16413) bde6a23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment