Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
to_datetime uses previous row's timezone when timezone not specified and utc=True #24992
Code Sample, a copy-pastable example if possible
pd.to_datetime([ '2018-11-28T00:00:00', '2018-11-28T00:00:00+12:00', '2018-11-28T00:00:00', '2018-11-28T00:00:00+06:00', '2018-11-28T00:00:00' ])
pd.to_datetime([ '2018-11-28T00:00:00', '2018-11-28T00:00:00+12:00', '2018-11-28T00:00:00', '2018-11-28T00:00:00+06:00', '2018-11-28T00:00:00' ], utc=True)
The behavior leads to unexpected and inconsistent treatment of datetimes where the timezone is not explicitly specified. Simply using the last timezone it saw from a datetime where the timezone was specified is pretty confusing.
I would be fine with it just assuming that any datetimes where the timezone is NOT specified are already in UTC. Another valid way of handling it would be to use os.environ['TZ']. Another valid way would be to add another input argument to to_datetime that allowed specifying which timezone to use when there is no timezone specified. Any of these would lead to conclusive, explainable results. This is what I would like to see:
In : items = ['2018-11-28T00:00:00+12:00', '2018-11-28T00:00:00'] In : a = pd.to_datetime(items, utc=True) In : b = pd.to_datetime(list(reversed(items)), utc=True)[::-1] In : a == b Out: array([ True, False])
The 0.24.0 behavior is the same as 0.23.4.