Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: Use is_utc check to improve performance of dateutil UTC in DatetimeIndex methods #23772

Merged
merged 9 commits into from Nov 18, 2018

Conversation

Projects
None yet
4 participants
@mroeschke
Copy link
Member

commented Nov 18, 2018

  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

Benchmarks from timeseries.py and timestamp.py

       before           after         ratio
     [91d1c503]       [9cc84951]
-      32.6±0.7μs       6.89±0.2μs     0.21  timestamp.TimestampOps.time_normalize(tzutc())

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.

       before           after         ratio
     [91d1c503]       [9cc84951]
-      4.72±0.1μs      4.26±0.06μs     0.90  timeseries.DatetimeIndex.time_get('dst')
-      5.09±0.4ms      2.79±0.02ms     0.55  timeseries.DatetimeAccessor.time_dt_accessor_normalize(tzutc())
-        297±20μs       42.9±0.2μs     0.14  timeseries.TzLocalize.time_infer_dst(tzutc())

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
@pep8speaks

This comment has been minimized.

Copy link

commented Nov 18, 2018

Hello @mroeschke! Thanks for submitting the PR.

Matt Roeschke

@jreback jreback added this to the 0.24.0 milestone Nov 18, 2018

@jreback

This comment has been minimized.

Copy link
Contributor

commented Nov 18, 2018

lgtm. I bet this actually fixed a bug where a dateutil UTC value was not treated correctly?

@jbrockmendel

This comment has been minimized.

Copy link
Member

commented Nov 18, 2018

Also datetime.timezones.utc in py3

@codecov

This comment has been minimized.

Copy link

commented Nov 18, 2018

Codecov Report

Merging #23772 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #23772      +/-   ##
==========================================
- Coverage   92.24%   92.24%   -0.01%     
==========================================
  Files         161      161              
  Lines       51433    51431       -2     
==========================================
- Hits        47446    47444       -2     
  Misses       3987     3987
Flag Coverage Δ
#multiple 90.64% <100%> (-0.01%) ⬇️
#single 42.28% <14.28%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/datetimes.py 96.2% <100%> (-0.01%) ⬇️
pandas/core/arrays/datetimes.py 98.5% <100%> (-0.01%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91d1c50...ca60770. Read the comment docs.

@jreback jreback merged commit 84cc0fb into pandas-dev:master Nov 18, 2018

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
pandas-dev.pandas Build #20181118.80 succeeded
Details
@jreback

This comment has been minimized.

Copy link
Contributor

commented Nov 18, 2018

thanks @mroeschke

as above, if you find some issues that now are closable because of the better checking (mine and @jbrockmendel comments). pls issue a PR with tests.

@mroeschke mroeschke deleted the mroeschke:dateutil_tz_performance branch Nov 18, 2018

thoo added a commit to thoo/pandas that referenced this pull request Nov 19, 2018

Merge remote-tracking branch 'upstream/master' into io_csv_docstring_…
…fixed

* upstream/master: (46 commits)
  DEPS: bump xlrd min version to 1.0.0 (pandas-dev#23774)
  BUG: Don't warn if default conflicts with dialect (pandas-dev#23775)
  BUG: Fixing memory leaks in read_csv (pandas-dev#23072)
  TST: Extend datetime64 arith tests to array classes, fix several broken cases (pandas-dev#23771)
  STYLE: Specify bare exceptions in pandas/tests (pandas-dev#23370)
  ENH: between_time, at_time accept axis parameter (pandas-dev#21799)
  PERF: Use is_utc check to improve performance of dateutil UTC in DatetimeIndex methods (pandas-dev#23772)
  CLN: io/formats/html.py: refactor (pandas-dev#22726)
  API: Make Categorical.searchsorted returns a scalar when supplied a scalar (pandas-dev#23466)
  TST: Add test case for GH14080 for overflow exception (pandas-dev#23762)
  BUG: Don't extract header names if none specified (pandas-dev#23703)
  BUG: Index.str.partition not nan-safe (pandas-dev#23558) (pandas-dev#23618)
  DEPR: tz_convert in the Timestamp constructor (pandas-dev#23621)
  PERF: Datetime/Timestamp.normalize for timezone naive datetimes (pandas-dev#23634)
  TST: Use new arithmetic fixtures, parametrize many more tests (pandas-dev#23757)
  REF/TST: Add more pytest idiom to parsers tests (pandas-dev#23761)
  DOC: Add ignore-deprecate argument to validate_docstrings.py (pandas-dev#23650)
  ENH: update pandas-gbq to 0.8.0, adds credentials arg (pandas-dev#23662)
  DOC: Improve error message to show correct order (pandas-dev#23652)
  ENH: Improve error message for empty object array (pandas-dev#23718)
  ...

Pingviinituutti added a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

Pingviinituutti added a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.