Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DatetimeArray._from_sequence with mixture of tz-naive and tz-aware data. #24569

Closed
TomAugspurger opened this issue Jan 2, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@TomAugspurger
Copy link
Contributor

commented Jan 2, 2019

This should probably raise

In [1]: import pandas as pd; import numpy as np

In [2]: arr = np.array([pd.Timestamp('2000'), pd.Timestamp('2000', tz='CET')])

In [3]: pd.arrays.DatetimeArray._from_sequence(arr)
Out[3]:
<DatetimeArrayMixin>
['2000-01-01 01:00:00+01:00', '2000-01-01 00:00:00+01:00']
Length: 2, dtype: datetime64[ns, CET]

For reference, a mix of different timezones does raise

In [4]: mix = np.array([pd.Timestamp('2000', tz="US/Central"), pd.Timestamp('2000', tz='CET')])

In [5]: pd.arrays.DatetimeArray._from_sequence(mix)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/sandbox/pandas/pandas/core/arrays/datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
   1753         try:
-> 1754             values, tz_parsed = conversion.datetime_to_datetime64(data)
   1755             # If tzaware, these values represent unix timestamps, so we

~/sandbox/pandas/pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.datetime_to_datetime64()
    178                     if not tz_compare(val.tzinfo, inferred_tz):
--> 179                         raise ValueError('Array must be all same time zone')
    180                 else:

ValueError: Array must be all same time zone

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-5-b2c990c41b55> in <module>
----> 1 pd.arrays.DatetimeArray._from_sequence(mix)

~/sandbox/pandas/pandas/core/arrays/datetimes.py in _from_sequence(cls, data, dtype, copy, tz, freq, dayfirst, yearfirst, ambiguous)
    326         subarr, tz, inferred_freq = sequence_to_dt64ns(
    327             data, dtype=dtype, copy=copy, tz=tz,
--> 328             dayfirst=dayfirst, yearfirst=yearfirst, ambiguous=ambiguous)
    329
    330         freq, freq_infer = dtl.validate_inferred_freq(freq, inferred_freq,

~/sandbox/pandas/pandas/core/arrays/datetimes.py in sequence_to_dt64ns(data, dtype, copy, tz, dayfirst, yearfirst, ambiguous)
   1660             #  or M8[ns] to denote wall times
   1661             data, inferred_tz = objects_to_datetime64ns(
-> 1662                 data, dayfirst=dayfirst, yearfirst=yearfirst)
   1663             tz = maybe_infer_tz(tz, inferred_tz)
   1664

~/sandbox/pandas/pandas/core/arrays/datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
   1757             return values.view('i8'), tz_parsed
   1758         except (ValueError, TypeError):
-> 1759             raise e
   1760
   1761     if tz_parsed is not None:

~/sandbox/pandas/pandas/core/arrays/datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
   1748             dayfirst=dayfirst,
   1749             yearfirst=yearfirst,
-> 1750             require_iso8601=require_iso8601
   1751         )
   1752     except ValueError as e:

~/sandbox/pandas/pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()
    458 @cython.wraparound(False)
    459 @cython.boundscheck(False)
--> 460 cpdef array_to_datetime(ndarray[object] values, str errors='raise',
    461                         bint dayfirst=False, bint yearfirst=False,
    462                         object utc=None, bint require_iso8601=False):

~/sandbox/pandas/pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()
    535                             iresult[i] = _ts.value
    536                         else:
--> 537                             raise ValueError('Tz-aware datetime.datetime '
    538                                              'cannot be converted to '
    539                                              'datetime64 unless utc=True')

ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True

@TomAugspurger TomAugspurger added this to the 0.24.0 milestone Jan 2, 2019

@jreback

This comment has been minimized.

Copy link
Contributor

commented Jan 2, 2019

I @jbrockmendel already noted this, its in to_datetime, but really because these hit different paths in array_to_datetime

In [1]: arr = np.array([pd.Timestamp('2000'), pd.Timestamp('2000', tz='CET')])
In [2]: pd.to_datetime(arr)
Out[2]: DatetimeIndex(['2000-01-01 01:00:00+01:00', '2000-01-01 00:00:00+01:00'], dtype='datetime64[ns, CET]', freq=None)

In [3]: mix = np.array([pd.Timestamp('2000', tz="US/Central"), pd.Timestamp('2000', tz='CET')])

In [4]: pd.to_datetime(mix)
ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True
@jbrockmendel

This comment has been minimized.

Copy link
Member

commented Jan 2, 2019

Yah, #24006 would handle these correctly, needs to come off the back-burner.

IIRC there was a "M8ns/i8 are treated inconsistently" issue in there too.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor Author

commented Jan 7, 2019

This isn't a blocker for 0.24.0 I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.