Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: Deprecate passing range-like arguments to DatetimeIndex, TimedeltaIndex #23919

Merged
merged 21 commits into from Nov 28, 2018

Conversation

Projects
None yet
7 participants
@jbrockmendel
Copy link
Member

commented Nov 26, 2018

Also verify_integrity since allowing that to be False complicates things, and internally if its False we should be using simple_new anyway

  • closes #20535
  • tests added / passed
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry
@pep8speaks

This comment has been minimized.

Copy link

commented Nov 26, 2018

Hello @jbrockmendel! Thanks for submitting the PR.

@jorisvandenbossche
Copy link
Member

left a comment

Thanks! Looks good to me, a minor comment.

Further, can you add an assertion for the warning on verify_integrity ?

Also, small suggestion: you now left one DatetimeIndex in there with an assert_produces_warning. I would maybe do such an assertion in a separate test (now that assertion is a bit buried in a bigger test that is for the rest using date_range)

Show resolved Hide resolved doc/source/whatsnew/v0.24.0.rst Outdated
@@ -385,7 +385,7 @@ def test_groupby_groups_datetimeindex(self):
groups = grouped.groups
assert isinstance(list(groups.keys())[0], datetime)

# GH 11442
# GH#11442

This comment has been minimized.

Copy link
@jorisvandenbossche

jorisvandenbossche Nov 26, 2018

Member

not important, but was just wondering: why are you adding the '#' everywhere?

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 26, 2018

Author Member

Mostly for internal consistency (not a big deal, but for grepping purposes). A little bit because I was curious how long it would take before someone asked about it.

This comment has been minimized.

Copy link
@jreback

jreback Nov 27, 2018

Contributor

we have have the gh-xxxx style as well, slight prefernce for that

with warnings.catch_warnings():
# we ignore warnings from passing verify_integrity=False
# TODO: If we knew what was going in to **d, we might be able to
# go through _simple_new instead

This comment has been minimized.

Copy link
@jorisvandenbossche

jorisvandenbossche Nov 26, 2018

Member

you can check __reduce__, d should be _data and the attributes dict

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 26, 2018

Author Member

good idea, that seems to work; just pushed

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

Looks like this broke the legacy pickle tests; reverted.

@codecov

This comment has been minimized.

Copy link

commented Nov 26, 2018

Codecov Report

❗️ No coverage uploaded for pull request base (master@30c1290). Click here to learn what that means.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master   #23919   +/-   ##
=========================================
  Coverage          ?   92.31%           
=========================================
  Files             ?      161           
  Lines             ?    51489           
  Branches          ?        0           
=========================================
  Hits              ?    47533           
  Misses            ?     3956           
  Partials          ?        0
Flag Coverage Δ
#multiple 90.71% <100%> (?)
#single 42.43% <53.33%> (?)
Impacted Files Coverage Δ
pandas/util/testing.py 86.09% <100%> (ø)
pandas/core/resample.py 96.99% <100%> (ø)
pandas/io/packers.py 88.08% <100%> (ø)
pandas/core/indexes/timedeltas.py 89.44% <100%> (ø)
pandas/tseries/holiday.py 93.17% <100%> (ø)
pandas/core/indexes/datetimes.py 96.51% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30c1290...cc40717. Read the comment docs.

@jbrockmendel jbrockmendel force-pushed the jbrockmendel:gentle branch from 3486e69 to 43a52fc Nov 27, 2018

@@ -131,10 +132,18 @@ def __new__(cls, data=None, unit=None, freq=None, start=None, end=None,
periods=None, closed=None, dtype=None, copy=False,
name=None, verify_integrity=True):

if verify_integrity is not True:

This comment has been minimized.

Copy link
@jreback

jreback Nov 27, 2018

Contributor

why isn't the default of verify_integrity None?

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

Because by default we do verify the integrity of a passed frequency. Best guess as to initial motivation is that verify_integrity=False is kind of like fastpath=True and was never really intended to be user-facing.

This comment has been minimized.

Copy link
@jreback

jreback Nov 27, 2018

Contributor

right but aren't you deprecating it?

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

Yes. In the future it will just be set to verify_integrity=True at the top of __new__. (and when the time comes, in the TDA/DTA constructors it will always be True.

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

We still need the variable to exist because there are cases in which we can determine it is not necessary, in which case we can skip a potentially-expensive check.

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

I don't recall that (and dont see how it would work), but yah, not passing freq would make verify_integrity unnecessary.

This comment has been minimized.

Copy link
@jorisvandenbossche

jorisvandenbossche Nov 27, 2018

Member

We still need the variable to exist because there are cases in which we can determine it is not necessary, in which case we can skip a potentially-expensive check.

@jbrockmendel Isn't the end goal to actually remove the verify_integrity keyword from the Index constructors? (not just keep it with a fixed True value). Because otherwise, why are you deprecating verify_integrity=False if we actually want to use it ourselves?
In cases that we determine the check is not necessary, we can use _simple_new ?

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

Eventually verify-integrity will not be in the signature. The first line of the method will just define it to be True.

This comment has been minimized.

Copy link
@jorisvandenbossche

jorisvandenbossche Nov 27, 2018

Member

OK, but then as @jreback said, we should put it at None so we can also deprecate the case for somebody setting it to True explicitly.

This comment has been minimized.

Copy link
@jbrockmendel

jbrockmendel Nov 27, 2018

Author Member

done

@@ -385,7 +385,7 @@ def test_groupby_groups_datetimeindex(self):
groups = grouped.groups
assert isinstance(list(groups.keys())[0], datetime)

# GH 11442
# GH#11442

This comment has been minimized.

Copy link
@jreback

jreback Nov 27, 2018

Contributor

we have have the gh-xxxx style as well, slight prefernce for that

@jreback

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2018

lgtm

@jbrockmendel

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2018

circle fail is Hypothesis; de-facto green

@mroeschke

This comment has been minimized.

Copy link
Member

commented Nov 28, 2018

Shouldn't we do the same for PeriodIndex as well?

@jbrockmendel

This comment has been minimized.

Copy link
Member Author

commented Nov 28, 2018

Shouldn't we do the same for PeriodIndex as well?

Possibly. There are also fields kwargs there we might want to deprecate. Prefer to do this separately.

@jreback

This comment has been minimized.

Copy link
Contributor

commented Nov 28, 2018

@jreback jreback added this to the 0.24.0 milestone Nov 28, 2018

@jorisvandenbossche jorisvandenbossche merged commit 6b3490f into pandas-dev:master Nov 28, 2018

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
pandas-dev.pandas Build #20181128.63 succeeded
Details
@jorisvandenbossche

This comment has been minimized.

Copy link
Member

commented Nov 28, 2018

thanks!

@jbrockmendel jbrockmendel deleted the jbrockmendel:gentle branch Nov 28, 2018

Pingviinituutti added a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

Pingviinituutti added a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.