Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DateTimeIndex.to_frame() drops timezone in pandas 0.24 #25809

Closed
jakevdp opened this issue Mar 20, 2019 · 3 comments

Comments

@jakevdp
Copy link
Contributor

commented Mar 20, 2019

Behavior in pandas 0.23:

>>> import pandas as pd
>>> pd.__version__
0.23.0
>>> x = pd.date_range(start='2019-01-01', end='2019-01-30', freq='D')
>>> x = x.tz_localize('UTC')
>>> x.to_frame().dtypes
0    datetime64[ns, UTC]
dtype: object

Behavior in pandas 0.24

>>> import pandas as pd
>>> pd.__version__
0.24.2
>>> x = pd.date_range(start='2019-01-01', end='2019-01-30', freq='D')
>>> x = x.tz_localize('UTC')
>>> x.to_frame().dtypes
0    datetime64[ns]
dtype: object
@jakevdp

This comment has been minimized.

Copy link
Contributor Author

commented Mar 20, 2019

Issue is here:

result = DataFrame({name: self.values.copy()})

index.values converts data to a numpy array with no timezone information in the dtype:

x = pd.date_range(start='2019-01-01', end='2019-01-30', freq='D')
x = x.tz_localize('UTC')
x.values.dtype
# dtype('<M8[ns]')
@jakevdp

This comment has been minimized.

Copy link
Contributor Author

commented Mar 21, 2019

Some initial experiments suggest that replacing self.values with self.to_numpy() will fix this; I'll give this a try and prepare a PR.

@mroeschke

This comment has been minimized.

Copy link
Member

commented Mar 21, 2019

Thanks for the report @jakevdp! I already have a PR up to fix this issue #25811.

The semantics with .values changed recently along with our migration to use ExtensionArrays more internally (in this case DatetimeArray). Passing the internal extension array (._values) into the constructor solves this issue.

@jreback jreback added this to the 0.25.0 milestone Mar 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.