Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timestamp.replace should handle ambiguous DST times gracefully #25017

Closed
ali-tny opened this issue Jan 30, 2019 · 2 comments

Comments

Projects
None yet
3 participants
@ali-tny
Copy link

commented Jan 30, 2019

>>> tz = pytz.timezone('Europe/London')
>>> d = datetime.datetime(2019, 10, 27, 1, 0)
>>> d = tz.localize(d, is_dst=True)
>>> ts = pd.Timestamp(d)
>>> ts == d 
True
>>> pd.Timestamp(d).replace(minute=0) == d.replace(minute=0)
False
>>> pd.Timestamp(d).replace(minute=0).tzinfo
<DstTzInfo 'Europe/London' GMT0:00:00 STD>
>>> d.replace(minute=0).tzinfo
<DstTzInfo 'Europe/London' BST+1:00:00 DST>

This is related to issue 18319 and the associated PR which makes pandas.Timestamp.replace behaviour deviate from datetime.datetime.replace when crossing DST transitions, and generally hand them more gracefully.
In cases when replacing into an ambiguous time (as above, when the clocks go back and repeat the same time in local time), the default behaviour of pandas is to normalize the datetime, using pytz's default setting is_dst=False. It would be useful to

  • have a way of specifying whether to go into DST or not and override the pytz default
  • in the case of replacing increments less than an hour, to copy the DST info of the original datetime (since the new time could be assumed to be in the same local hour)
@mroeschke

This comment has been minimized.

Copy link
Member

commented Jan 30, 2019

Note Timestamp.replace has a fold argument like the PY3.6 version; however, it's just not implemented yet. It just needs to be passed into localize as is_dst=bool(fold).

@ali-tny

This comment has been minimized.

Copy link
Author

commented Jan 30, 2019

For ambiguous times I believe you would want is_dst=not bool(fold) since fold = 0 refers to the first moment chronologically that time is seen on the clock locally, aka when DST is still in place

@jreback jreback added this to the 0.25.0 milestone Jan 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.