New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datetime like series .dt properties fail after ts.dt.tz_localize(tz1).dt.tz_convert(tz2) sequence #11295

Closed
witosx opened this Issue Oct 12, 2015 · 2 comments

Comments

Projects
None yet
3 participants
@witosx

witosx commented Oct 12, 2015

Second tz_convert fails with AmbiguousTimeError ('UTC' in the example, but other timezones fail too).

EDIT: Actually it's not only second tz_convert that fails, it looks like most (if not all) of the .dt accessed properties are affected

t = pd.Series(pd.date_range('2015-01-01', '2016-01-01', freq='T'))
t = t.dt.tz_localize('UTC').dt.tz_convert('America/Chicago')
t.head(10)
0   2014-12-31 18:00:00-06:00
1   2014-12-31 18:01:00-06:00
2   2014-12-31 18:02:00-06:00
3   2014-12-31 18:03:00-06:00
4   2014-12-31 18:04:00-06:00
5   2014-12-31 18:05:00-06:00
6   2014-12-31 18:06:00-06:00
7   2014-12-31 18:07:00-06:00
8   2014-12-31 18:08:00-06:00
9   2014-12-31 18:09:00-06:00
dtype: datetime64[ns, America/Chicago]
t.dt.tz_convert('UTC')
---------------------------------------------------------------------------

AmbiguousTimeError                        Traceback (most recent call last)

/usr/lib/python3.5/site-packages/pandas/core/series.py in _make_dt_accessor(self)
   2639         try:
-> 2640             return maybe_to_datetimelike(self)
   2641         except Exception:


/usr/lib/python3.5/site-packages/pandas/tseries/common.py in maybe_to_datetimelike(data, copy)
     48     if is_datetime64_dtype(data.dtype) or is_datetime64tz_dtype(data.dtype):
---> 49         return DatetimeProperties(DatetimeIndex(data, copy=copy, freq='infer'), index, name=data.name)
     50     elif is_timedelta64_dtype(data.dtype):


/usr/lib/python3.5/site-packages/pandas/util/decorators.py in wrapper(*args, **kwargs)
     88                     kwargs[new_arg_name] = new_arg_value
---> 89             return func(*args, **kwargs)
     90         return wrapper


/usr/lib/python3.5/site-packages/pandas/tseries/index.py in __new__(cls, data, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, closed, ambiguous, dtype, **kwargs)
    343                     subarr = tslib.tz_localize_to_utc(ints, tz,
--> 344                                                       ambiguous=ambiguous)
    345 


pandas/tslib.pyx in pandas.tslib.tz_localize_to_utc (pandas/tslib.c:64516)()


AmbiguousTimeError: Cannot infer dst time from Timestamp('2015-11-01 01:00:00'), try using the 'ambiguous' argument


During handling of the above exception, another exception occurred:


AttributeError                            Traceback (most recent call last)

<ipython-input-2-eb75730cd6fc> in <module>()
----> 1 t.dt.tz_convert('UTC')


/usr/lib/python3.5/site-packages/pandas/core/generic.py in __getattr__(self, name)
   2239                 or name in self._metadata
   2240                 or name in self._accessors):
-> 2241             return object.__getattribute__(self, name)
   2242         else:
   2243             if name in self._info_axis:


/usr/lib/python3.5/site-packages/pandas/core/base.py in __get__(self, instance, owner)
    186             # this ensures that Series.str.<method> is well defined
    187             return self.accessor_cls
--> 188         return self.construct_accessor(instance)
    189 
    190     def __set__(self, instance, value):


/usr/lib/python3.5/site-packages/pandas/core/series.py in _make_dt_accessor(self)
   2640             return maybe_to_datetimelike(self)
   2641         except Exception:
-> 2642             raise AttributeError("Can only use .dt accessor with datetimelike "
   2643                                  "values")
   2644 


AttributeError: Can only use .dt accessor with datetimelike values

@witosx witosx changed the title from datetime like series second tz_convert fails to datetime like series .dt properties fail after ts.dt.tz_localize(tz1).dt.tz_convert(tz2) sequence Oct 12, 2015

@jorisvandenbossche jorisvandenbossche added this to the 0.17.1 milestone Oct 12, 2015

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Oct 12, 2015

Member

This seems to come down to the failing of DatetimeIndex(t) (which is used within the dt accessor):

In [37]: t.head()
Out[37]:
0   2014-12-31 18:00:00-06:00
1   2014-12-31 18:01:00-06:00
2   2014-12-31 18:02:00-06:00
3   2014-12-31 18:03:00-06:00
4   2014-12-31 18:04:00-06:00
dtype: datetime64[ns, America/Chicago]

In [38]: pd.DatetimeIndex(t)
---------------------------------------------------------------------------
AmbiguousTimeError                        Traceback (most recent call last)
<ipython-input-38-3e6430dea298> in <module>()
----> 1 pd.DatetimeIndex(t)

c:\users\vdbosscj\scipy\pandas-joris\pandas\util\decorators.pyc in wrapper(*args
, **kwargs)
     87                 else:
     88                     kwargs[new_arg_name] = new_arg_value
---> 89             return func(*args, **kwargs)
     90         return wrapper
     91     return _deprecate_kwarg

c:\users\vdbosscj\scipy\pandas-joris\pandas\tseries\index.pyc in __new__(cls, da
ta, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, clos
ed, ambiguous, dtype, **kwargs)
    342                     ints = subarr.view('i8')
    343                     subarr = tslib.tz_localize_to_utc(ints, tz,
--> 344                                                       ambiguous=ambiguou
s)
    345
    346                 subarr = subarr.view(_NS_DTYPE)

tslib.pyx in pandas.tslib.tz_localize_to_utc (pandas\tslib.c:55271)()

AmbiguousTimeError: Cannot infer dst time from Timestamp('2015-11-01 01:00:00'),
 try using the 'ambiguous' argument

When providing tz data to DatetimeIndex, it first makes it naive and then converts back to the tz, which fails without supplying ambiguous='infer'.

Member

jorisvandenbossche commented Oct 12, 2015

This seems to come down to the failing of DatetimeIndex(t) (which is used within the dt accessor):

In [37]: t.head()
Out[37]:
0   2014-12-31 18:00:00-06:00
1   2014-12-31 18:01:00-06:00
2   2014-12-31 18:02:00-06:00
3   2014-12-31 18:03:00-06:00
4   2014-12-31 18:04:00-06:00
dtype: datetime64[ns, America/Chicago]

In [38]: pd.DatetimeIndex(t)
---------------------------------------------------------------------------
AmbiguousTimeError                        Traceback (most recent call last)
<ipython-input-38-3e6430dea298> in <module>()
----> 1 pd.DatetimeIndex(t)

c:\users\vdbosscj\scipy\pandas-joris\pandas\util\decorators.pyc in wrapper(*args
, **kwargs)
     87                 else:
     88                     kwargs[new_arg_name] = new_arg_value
---> 89             return func(*args, **kwargs)
     90         return wrapper
     91     return _deprecate_kwarg

c:\users\vdbosscj\scipy\pandas-joris\pandas\tseries\index.pyc in __new__(cls, da
ta, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, clos
ed, ambiguous, dtype, **kwargs)
    342                     ints = subarr.view('i8')
    343                     subarr = tslib.tz_localize_to_utc(ints, tz,
--> 344                                                       ambiguous=ambiguou
s)
    345
    346                 subarr = subarr.view(_NS_DTYPE)

tslib.pyx in pandas.tslib.tz_localize_to_utc (pandas\tslib.c:55271)()

AmbiguousTimeError: Cannot infer dst time from Timestamp('2015-11-01 01:00:00'),
 try using the 'ambiguous' argument

When providing tz data to DatetimeIndex, it first makes it naive and then converts back to the tz, which fails without supplying ambiguous='infer'.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Oct 12, 2015

Contributor

@witosx give #11301 a try. This was passing thru the ambiguous kw incorrectly.

Contributor

jreback commented Oct 12, 2015

@witosx give #11301 a try. This was passing thru the ambiguous kw incorrectly.

jreback added a commit to jreback/pandas that referenced this issue Oct 13, 2015

jreback added a commit that referenced this issue Oct 13, 2015

Merge pull request #11301 from jreback/tz_convert
BUG: Bug in tz-conversions with an ambiguous time and .dt accessors, #11295
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment