BUG/API: Index creation with different tz coerces DatetimeIndex #11696

sinhrks · Nov 24, 2015

Based on the type check added in #11588, the change affects to fewer functions than I originally thought.

I think this might fail with NaT

I think you can ignore if this is None (meaning its a NaT)

I also don't think you actually need to use get_timezone, just check object equality no? (e.g. that the timezones are all the same object)

I would also move this to inference.pyx I think

get_timezone is needed to handle DST properly:

import pandas as pd
t1 = pd.Timestamp('2011-01-01', tz='US/Eastern')
t2 = pd.Timestamp('2011-08-01', tz='US/Eastern')

t1.tz == t2.tz
# False

from pandas.tslib import get_timezone
get_timezone(t1.tz) == get_timezone(t2.tz)
# True

hmm ok

do an equality check on tz first then only check get_timezone if it's False (of course if that's False then return)

I think this would be much more performant

is my comment above worth implementing? is this performant enough? (e.g. I dont' think get_timezone is particualy performant when called lots of times....)

was this worth it?

sinhrks · Nov 28, 2015

Thanks, moved to inferrence.pyx. Also fixed and added tests for NaT.

jreback · Nov 29, 2015

lgtm. comment above.

sinhrks · Dec 10, 2015

Thanks, I've fixed the tz comparison logic based on your comment below.

do an equality check on tz first then only check get_timezone if it's False

jreback · Dec 10, 2015

merged via b80b5c7

thanks!

sinhrks added API Design Timezones labels Nov 24, 2015

sinhrks added this to the 0.18.0 milestone Nov 24, 2015

sinhrks BUG/API: Index creation with different tz coerces DatetimeIndex

162bd1d

jreback closed this Dec 10, 2015

jreback referenced this pull request Dec 10, 2015
Closed
API: DatetimeIndex creation with mixed tz timestamps #11488

sinhrks deleted the sinhrks:index_new_tz branch Dec 10, 2015

sinhrks referenced this pull request Jul 12, 2016
Open
API: Datetime tz-aware/tz-naive ops consistency #11456
3 of 9 tasks complete

@@ -1409,6 +1409,26 @@ cpdef inline object maybe_get_tz(object tz):
          return tz
+    +cpdef is_datetime_with_singletz_array(ndarray[object] values):
+    +    """
+    +    Check values have the same tzinfo attribute.
+    +    Doesn't check values are datetime-like types.
+    +    """
+    +    cdef Py_ssize_t i, n = len(values)
+    +    cdef object base_tz, tz
+    +
+    +    if n == 0:
+    +        return False
+    +
+    +    base_tz = getattr(values[0], 'tzinfo', None)
+    +    base_tz = get_timezone(base_tz)

+    +    """
+    +    Check values have the same tzinfo attribute.
+    +    Doesn't check values are datetime-like types.
+    +    """
+    +    cdef Py_ssize_t i, n = len(values)
+    +    cdef object base_tz, tz
+    +
+    +    if n == 0:
+    +        return False
+    +
+    +    base_tz = getattr(values[0], 'tzinfo', None)
+    +    base_tz = get_timezone(base_tz)
+    +
+    +    for i in range(1, n):
+    +        tz = getattr(values[i], 'tzinfo', None)
+    +        if base_tz != get_timezone(tz):

+    +    Doesn't check values are datetime-like types.
+    +    """
+    +    cdef Py_ssize_t i, n = len(values)
+    +    cdef object base_tz, tz
+    +
+    +    if n == 0:
+    +        return False
+    +
+    +    base_tz = getattr(values[0], 'tzinfo', None)
+    +    base_tz = get_timezone(base_tz)
+    +
+    +    for i in range(1, n):
+    +        tz = getattr(values[i], 'tzinfo', None)
+    +        if base_tz != get_timezone(tz):
+    +            return False
+    +    return True

pandas-dev/pandas

BUG/API: Index creation with different tz coerces DatetimeIndex #11696

sinhrks commented Nov 24, 2015

sinhrks added API Design Timezones labels Nov 24, 2015

sinhrks added this to the 0.18.0 milestone Nov 24, 2015

jreback commented on an outdated diff Nov 27, 2015

jreback commented on an outdated diff Nov 27, 2015

jreback and 1 other commented on an outdated diff Nov 27, 2015

sinhrks commented Nov 28, 2015

jreback commented Nov 29, 2015

sinhrks commented Dec 10, 2015

jreback commented Dec 10, 2015

jreback closed this Dec 10, 2015

jreback referenced this pull request Dec 10, 2015

API: DatetimeIndex creation with mixed tz timestamps #11488

sinhrks deleted the sinhrks:index_new_tz branch Dec 10, 2015

sinhrks referenced this pull request Jul 12, 2016

API: Datetime tz-aware/tz-naive ops consistency #11456

pandas-dev/pandas

Join GitHub today

BUG/API: Index creation with different tz coerces DatetimeIndex #11696

Conversation

sinhrks commented Nov 24, 2015

sinhrks added API Design Timezones labels Nov 24, 2015

sinhrks added this to the 0.18.0 milestone Nov 24, 2015

Show 1 comment Hide comment jreback commented on an outdated diff Nov 27, 2015

Show 1 comment Hide comment jreback commented on an outdated diff Nov 27, 2015

Show 6 comments Hide comments jreback and 1 other commented on an outdated diff Nov 27, 2015

sinhrks commented Nov 28, 2015

jreback commented Nov 29, 2015

sinhrks commented Dec 10, 2015

jreback commented Dec 10, 2015

jreback closed this Dec 10, 2015

jreback referenced this pull request Dec 10, 2015

sinhrks deleted the sinhrks:index_new_tz branch Dec 10, 2015

sinhrks referenced this pull request Jul 12, 2016

jreback commented on an outdated diff Nov 27, 2015

jreback commented on an outdated diff Nov 27, 2015

jreback and 1 other commented on an outdated diff Nov 27, 2015