Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API/ENH: tz_localize handling of nonexistent times: rename keyword + add shift option #22644

Merged
merged 64 commits into from Oct 25, 2018
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
bf5e7bf
ENH: Add handling of nonexistent times
Sep 4, 2018
36d13c7
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 5, 2018
a5ea445
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 7, 2018
8753d00
correct misspelling
Sep 7, 2018
e1a6c6a
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 7, 2018
a7c86c8
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 8, 2018
1884c7b
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 8, 2018
a6a05df
change method of handling nonexistent times
Sep 8, 2018
c4dc8aa
Add another comment
Sep 8, 2018
1bc81db
Add tests for timestamps
Sep 9, 2018
c81d58c
Add tests for datetimeindex
Sep 9, 2018
b2c8429
Add series test and entry in timeseries.rst
Sep 9, 2018
a65987d
Add whatsnew
Sep 9, 2018
710014c
Clean up docstring
Sep 9, 2018
93159e5
Fix nat doc
Sep 9, 2018
a0ffcdd
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 11, 2018
219256f
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 11, 2018
d435481
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 14, 2018
7c849b6
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 14, 2018
56ac4fe
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 14, 2018
b7b09bd
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 19, 2018
94a72a5
add versionadded
Sep 19, 2018
39b769e
Remove whitespace
Sep 19, 2018
18664d8
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 24, 2018
8852d43
Depreciate errors and see what needs warning captures
Sep 24, 2018
38b95e9
Correct NaT docstring
Sep 24, 2018
c88b0d8
edit whatsnew and check for raised DeprecationWarning
Sep 24, 2018
1bae682
Address review
Sep 26, 2018
d30f891
change default errors argument to None
Sep 26, 2018
f337692
Map depreciation correctly and test
Sep 26, 2018
6a12a7e
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 26, 2018
a7b8357
Try to correctly test for FutureWarning
Sep 26, 2018
7ad87ec
Try adjusting catching FutureWarning
Sep 27, 2018
abad726
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 27, 2018
6be1c25
Reorder context managers
Sep 27, 2018
f8be4b6
clear previously seen FutureWarning
Sep 28, 2018
c192c9f
separate test
Sep 28, 2018
8909f38
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 28, 2018
49f203f
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 30, 2018
01678c7
adjust test
Sep 30, 2018
707fdde
Merge remote-tracking branch 'upstream/master' into normalize_tz
Sep 30, 2018
ae27a50
Remove errors argument to tz_localize_to_utc
Sep 30, 2018
85ed25e
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 3, 2018
9041ebe
Add nonexistent assert
Oct 4, 2018
a4cdac2
Handle default None arg
Oct 5, 2018
0a9c1db
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 5, 2018
efb382e
Address review
Oct 6, 2018
61c73ca
Catch another warning
Oct 6, 2018
20cc925
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 7, 2018
394a0db
Add extra docstring
Oct 7, 2018
a5253ee
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 8, 2018
5185683
Edit whatsnew
Oct 8, 2018
ba1bfed
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 11, 2018
8b06c96
Address comments
Oct 11, 2018
42ae923
Remove stacklevel
Oct 12, 2018
fe575fe
Add back check_stacklevel
Oct 12, 2018
3482f92
Add blank line for rendering
Oct 17, 2018
f0e43e2
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 18, 2018
b98d4cf
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 18, 2018
e6c5b2d
Validate nonexistent argument
Oct 18, 2018
83423ad
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 19, 2018
1ca0ab2
Fix type
Oct 19, 2018
5bcc977
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 24, 2018
8cf16e2
Merge remote-tracking branch 'upstream/master' into normalize_tz
Oct 24, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions pandas/_libs/tslibs/nattype.pyx
Expand Up @@ -565,6 +565,8 @@ class NaTType(_NaT):
- 'raise' will raise an AmbiguousTimeError for an ambiguous time

nonexistent : 'shift', 'NaT', default 'raise'
A nonexistent time doesn't not exist in a particular timezone
where clocks moved forward due to DST.
- 'shift' will shift the nonexistent time forward to the closest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, rst formatting nitpick: there needs to be a blank line between the first sentences, and the start of this list ... (getting rst right can be annoying ..)

existing time
- 'NaT' will return NaT where there are nonexistent times
jorisvandenbossche marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
5 changes: 3 additions & 2 deletions pandas/_libs/tslibs/timestamps.pyx
Expand Up @@ -980,6 +980,8 @@ class Timestamp(_Timestamp):
- 'raise' will raise an AmbiguousTimeError for an ambiguous time

nonexistent : 'shift', 'NaT', default 'raise'
jorisvandenbossche marked this conversation as resolved.
Show resolved Hide resolved
A nonexistent time doesn't not exist in a particular timezone
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A nonexistent time doesn't not exist in a particular timezone
A nonexistent time does not exist in a particular timezone

where clocks moved forward due to DST.
- 'shift' will shift the nonexistent time forward to the closest
existing time
- 'NaT' will return NaT where there are nonexistent times
Expand Down Expand Up @@ -1013,8 +1015,7 @@ class Timestamp(_Timestamp):
warnings.warn("The errors argument is deprecated and will be "
"removed in a future release. Use "
"nonexistent='NaT' or nonexistent='raise' "
"instead.", FutureWarning,
stacklevel=2)
"instead.", FutureWarning)
if errors == 'coerce':
nonexistent = 'NaT'
elif errors == 'raise':
Expand Down
5 changes: 3 additions & 2 deletions pandas/core/arrays/datetimes.py
Expand Up @@ -649,6 +649,8 @@ def tz_localize(self, tz, ambiguous='raise', nonexistent='raise',
times

nonexistent : 'shift', 'NaT' default 'raise'
A nonexistent time doesn't not exist in a particular timezone
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A nonexistent time doesn't not exist in a particular timezone
A nonexistent time does not exist in a particular timezone

where clocks moved forward due to DST.
- 'shift' will shift the nonexistent times forward to the closest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you reuse the explanation about what a non-existent time is that is in the errors keyword below?

(and idem for the other occurences of this in docstrings)

existing time
- 'NaT' will return NaT where there are nonexistent times
Expand Down Expand Up @@ -711,8 +713,7 @@ def tz_localize(self, tz, ambiguous='raise', nonexistent='raise',
warnings.warn("The errors argument is deprecated and will be "
"removed in a future release. Use "
"nonexistent='NaT' or nonexistent='raise' "
"instead.", FutureWarning,
stacklevel=2)
"instead.", FutureWarning)
if errors == 'coerce':
nonexistent = 'NaT'
elif errors == 'raise':
Expand Down
2 changes: 2 additions & 0 deletions pandas/core/generic.py
Expand Up @@ -8656,6 +8656,8 @@ def tz_localize(self, tz, axis=0, level=None, copy=True,
- 'raise' will raise an AmbiguousTimeError if there are ambiguous
times
nonexistent : 'shift', 'NaT', default 'raise'
A nonexistent time doesn't not exist in a particular timezone
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A nonexistent time doesn't not exist in a particular timezone
A nonexistent time does not exist in a particular timezone

where clocks moved forward due to DST.
- 'shift' will shift the nonexistent times forward to the closest
existing time
- 'NaT' will return NaT where there are nonexistent times
Expand Down
8 changes: 4 additions & 4 deletions pandas/tests/indexes/datetimes/test_timezones.py
Expand Up @@ -312,12 +312,12 @@ def test_dti_tz_localize_nonexistent_raise_coerce(self):
index.tz_localize(tz=tz)

with pytest.raises(pytz.NonExistentTimeError):
with tm.assert_produces_warning(FutureWarning,
check_stacklevel=False):
with tm.assert_produces_warning(FutureWarning):
index.tz_localize(tz=tz, errors='raise')

with tm.assert_produces_warning(FutureWarning, check_stacklevel=False,
clear=FutureWarning):
with tm.assert_produces_warning(FutureWarning,
clear=FutureWarning,
check_stacklevel=False):
result = index.tz_localize(tz=tz, errors='coerce')
test_times = ['2015-03-08 01:00-05:00', 'NaT',
'2015-03-08 03:00-04:00']
Expand Down
13 changes: 5 additions & 8 deletions pandas/tests/scalar/timestamp/test_timezones.py
Expand Up @@ -87,18 +87,16 @@ def test_tz_localize_nonexistent(self, stamp, tz):
ts.tz_localize(tz)
# GH 22644
with pytest.raises(NonExistentTimeError):
with tm.assert_produces_warning(FutureWarning,
check_stacklevel=False):
with tm.assert_produces_warning(FutureWarning):
ts.tz_localize(tz, errors='raise')
with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
with tm.assert_produces_warning(FutureWarning):
assert ts.tz_localize(tz, errors='coerce') is NaT

def test_tz_localize_errors_ambiguous(self):
# GH#13057
ts = Timestamp('2015-11-1 01:00')
with pytest.raises(AmbiguousTimeError):
with tm.assert_produces_warning(FutureWarning,
check_stacklevel=False):
with tm.assert_produces_warning(FutureWarning):
ts.tz_localize('US/Pacific', errors='coerce')

@pytest.mark.filterwarnings('ignore::FutureWarning')
Expand All @@ -107,16 +105,15 @@ def test_tz_localize_errors_invalid_arg(self):
tz = 'Europe/Warsaw'
ts = Timestamp('2015-03-29 02:00:00')
with pytest.raises(ValueError):
with tm.assert_produces_warning(FutureWarning,
check_stacklevel=False):
with tm.assert_produces_warning(FutureWarning):
ts.tz_localize(tz, errors='foo')

def test_tz_localize_errors_coerce(self):
# GH 22644
# make sure errors='coerce' gets mapped correctly to nonexistent
tz = 'Europe/Warsaw'
ts = Timestamp('2015-03-29 02:00:00')
with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
with tm.assert_produces_warning(FutureWarning):
result = ts.tz_localize(tz, errors='coerce')
expected = ts.tz_localize(tz, nonexistent='NaT')
assert result is expected
Expand Down