-
-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PERF: Datetime/Timestamp.normalize for timezone naive datetimes #23634
Changes from 17 commits
62827ef
ff171f7
efb281f
cc6eee0
b90abd9
35fac22
f09559a
0b3a664
9042aa1
3a23170
52a7eb2
6204d21
1f1d455
44a8808
e6c74d2
0ece208
bc5571a
243d73a
fb11dcf
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,7 +8,7 @@ | |
from pandas._libs import lib, tslib | ||
from pandas._libs.tslib import Timestamp, NaT, iNaT | ||
from pandas._libs.tslibs import ( | ||
normalize_date, | ||
ccalendar, normalize_date, | ||
conversion, fields, timezones, | ||
resolution as libresolution) | ||
|
||
|
@@ -853,7 +853,14 @@ def normalize(self): | |
'2014-08-01 00:00:00+05:30'], | ||
dtype='datetime64[ns, Asia/Calcutta]', freq=None) | ||
""" | ||
new_values = conversion.normalize_i8_timestamps(self.asi8, self.tz) | ||
if self.tz is None or timezones.is_utc(self.tz): | ||
not_null = self.notnull() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should this be There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you use notna |
||
DAY_NS = ccalendar.DAY_SECONDS * 1000000000 | ||
new_values = self.asi8.copy() | ||
adjustment = (new_values[not_null] % DAY_NS) | ||
new_values[not_null] = new_values[not_null] - adjustment | ||
else: | ||
new_values = conversion.normalize_i8_timestamps(self.asi8, self.tz) | ||
return type(self)(new_values, freq='infer').tz_localize(self.tz) | ||
|
||
def to_period(self, freq=None): | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -328,6 +328,17 @@ def test_replace_dst_border(self): | |
expected = Timestamp('2013-11-3 03:00:00', tz='America/Chicago') | ||
assert result == expected | ||
|
||
# -------------------------------------------------------------- | ||
# Timestamp.normalize | ||
|
||
@pytest.mark.parametrize('arg', ['2013-11-30', '2013-11-30 12:00:00']) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is there a normalize_nat test as well? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't define There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could have one for Timstamp mirroring (another issue). Probably would just return |
||
def test_normalize(self, tz_naive_fixture, arg): | ||
tz = tz_naive_fixture | ||
ts = Timestamp(arg, tz=tz) | ||
result = ts.normalize() | ||
expected = Timestamp('2013-11-30', tz=tz) | ||
assert result == expected | ||
|
||
# -------------------------------------------------------------- | ||
|
||
@td.skip_if_windows | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this case never reached?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct. This case (the naive case) is handled in these two places now:
https://github.com/pandas-dev/pandas/pull/23634/files#diff-231ac35d2116a12844a7cfed02730580R1289
https://github.com/pandas-dev/pandas/pull/23634/files#diff-960da60b33d858481ad8799b0e84764bR835