Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Appending Series with dateutil timezone raises TypeError: unhashable type: 'tzutc' #23682

Closed
semigodking opened this issue Nov 14, 2018 · 2 comments

Comments

Projects
None yet
4 participants
@semigodking
Copy link

commented Nov 14, 2018

Code Sample, a copy-pastable example if possible

# Your code here
import pandas as pd
from dateutil.parser import parse as dt_parse

date = dt_parse('2018-10-24 07:30:00+00:00')
ser = pd.Series({'date': date})
ser2 = pd.Series({'date': date, 'a': 1.0, 'b': 2.0})
df = pd.DataFrame(columns=['c', 'd'])
# No exception
df2 = df.append(ser, ignore_index=True)
# Exception raised from next line
df2 = df.append(ser2, ignore_index=True)

Problem description

The last line of code above causes an exception.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-c167f10681a7> in <module>()
      1 ser2 = pd.Series({'date': date})
----> 2 df2 = df.append(ser, ignore_index=True)

~/vp/venv/lib/python3.5/site-packages/pandas/core/frame.py in append(self, other, ignore_index, verify_integrity)
   5155                               index=index,
   5156                               columns=combined_columns)
-> 5157             other = other._convert(datetime=True, timedelta=True)
   5158             if not self.columns.equals(combined_columns):
   5159                 self = self.reindex(columns=combined_columns)

~/vp/venv/lib/python3.5/site-packages/pandas/core/generic.py in _convert(self, datetime, numeric, timedelta, coerce, copy)
   4064             self._data.convert(datetime=datetime, numeric=numeric,
   4065                                timedelta=timedelta, coerce=coerce,
-> 4066                                copy=copy)).__finalize__(self)
   4067 
   4068     # TODO: Remove in 0.18 or 2017, which ever is sooner

~/vp/venv/lib/python3.5/site-packages/pandas/core/internals.py in convert(self, **kwargs)
   3458 
   3459     def convert(self, **kwargs):
-> 3460         return self.apply('convert', **kwargs)
   3461 
   3462     def replace(self, **kwargs):

~/vp/venv/lib/python3.5/site-packages/pandas/core/internals.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
   3322 
   3323             kwargs['mgr'] = self
-> 3324             applied = getattr(b, f)(**kwargs)
   3325             result_blocks = _extend_blocks(applied, result_blocks)
   3326 

~/vp/venv/lib/python3.5/site-packages/pandas/core/internals.py in convert(self, *args, **kwargs)
   2128 
   2129         if by_item and not self._is_single_block:
-> 2130             blocks = self.split_and_operate(None, f, False)
   2131         else:
   2132             values = f(None, self.values.ravel(), None)

~/vp/venv/lib/python3.5/site-packages/pandas/core/internals.py in split_and_operate(self, mask, f, inplace)
    476             # need a new block
    477             if m.any():
--> 478                 nv = f(m, v, i)
    479             else:
    480                 nv = v if inplace else v.copy()

~/vp/venv/lib/python3.5/site-packages/pandas/core/internals.py in f(m, v, i)
   2118         def f(m, v, i):
   2119             shape = v.shape
-> 2120             values = fn(v.ravel(), **fn_kwargs)
   2121             try:
   2122                 values = values.reshape(shape)

~/vp/venv/lib/python3.5/site-packages/pandas/core/dtypes/cast.py in soft_convert_objects(values, datetime, numeric, timedelta, coerce, copy)
    805     # Soft conversions
    806     if datetime:
--> 807         values = lib.maybe_convert_objects(values, convert_datetime=datetime)
    808 
    809     if timedelta and is_object_dtype(values.dtype):

pandas/_libs/src/inference.pyx in pandas._libs.lib.maybe_convert_objects()

TypeError: unhashable type: 'tzutc'

Expected Output

No exception.

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]
INSTALLED VERSIONS

commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-130-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_HK.UTF-8
LOCALE: en_HK.UTF-8

pandas: 0.21.0
pytest: None
pip: 9.0.1
setuptools: 36.2.7
Cython: None
numpy: 1.13.3
scipy: None
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.2.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0b10
sqlalchemy: 1.1.13
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@jbrockmendel

This comment has been minimized.

Copy link
Member

commented Nov 14, 2018

The traceback references a file that hasn’t existed for a while. Can you try this on master?

@mroeschke

This comment has been minimized.

Copy link
Member

commented Nov 14, 2018

I'm getting the same error on master but different traceback. I had posted an issue about hashability of dateutil timezones, but it appears there's good reason why they aren't yet. dateutil/dateutil#792

In [6]: pd.__version__
Out[6]: '0.24.0.dev0+997.ga197837c4'

In [7]: df2 = df.append(ser2, ignore_index=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-d913f5e96179> in <module>()
----> 1 df2 = df.append(ser2, ignore_index=True)

~/pandas-mroeschke/pandas/core/frame.py in append(self, other, ignore_index, verify_integrity, sort)
   6497                               index=index,
   6498                               columns=combined_columns)
-> 6499             other = other._convert(datetime=True, timedelta=True)
   6500             if not self.columns.equals(combined_columns):
   6501                 self = self.reindex(columns=combined_columns)

~/pandas-mroeschke/pandas/core/generic.py in _convert(self, datetime, numeric, timedelta, coerce, copy)
   5509             self._data.convert(datetime=datetime, numeric=numeric,
   5510                                timedelta=timedelta, coerce=coerce,
-> 5511                                copy=copy)).__finalize__(self)
   5512
   5513     def convert_objects(self, convert_dates=True, convert_numeric=False,

~/pandas-mroeschke/pandas/core/internals/managers.py in convert(self, **kwargs)
    535
    536     def convert(self, **kwargs):
--> 537         return self.apply('convert', **kwargs)
    538
    539     def replace(self, **kwargs):

~/pandas-mroeschke/pandas/core/internals/managers.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
    403                                             copy=align_copy)
    404
--> 405             applied = getattr(b, f)(**kwargs)
    406             result_blocks = _extend_blocks(applied, result_blocks)
    407

~/pandas-mroeschke/pandas/core/internals/blocks.py in convert(self, *args, **kwargs)
   2350
   2351         if by_item and not self._is_single_block:
-> 2352             blocks = self.split_and_operate(None, f, False)
   2353         else:
   2354             values = f(None, self.values.ravel(), None)

~/pandas-mroeschke/pandas/core/internals/blocks.py in split_and_operate(self, mask, f, inplace)
    500             # need a new block
    501             if m.any():
--> 502                 nv = f(m, v, i)
    503             else:
    504                 nv = v if inplace else v.copy()

~/pandas-mroeschke/pandas/core/internals/blocks.py in f(m, v, i)
   2340         def f(m, v, i):
   2341             shape = v.shape
-> 2342             values = fn(v.ravel(), **fn_kwargs)
   2343             try:
   2344                 values = values.reshape(shape)

~/pandas-mroeschke/pandas/core/dtypes/cast.py in soft_convert_objects(values, datetime, numeric, timedelta, coerce, copy)
    854         try:
    855             values = lib.maybe_convert_objects(values,
--> 856                                                convert_datetime=datetime)
    857         except OutOfBoundsDatetime:
    858             pass

~/pandas-mroeschke/pandas/_libs/lib.pyx in pandas._libs.lib.maybe_convert_objects()
   2046     # we try to coerce datetime w/tz but must all have the same tz
   2047     if seen.datetimetz_:
-> 2048         if len({getattr(val, 'tzinfo', None) for val in objects}) == 1:
   2049             from pandas import DatetimeIndex
   2050             return DatetimeIndex(objects)

TypeError: unhashable type: 'tzutc'

@mroeschke mroeschke changed the title TypeError: unhashable type: 'tzutc' BUG: Appending Series with dateutil timezone raises TypeError: unhashable type: 'tzutc' Nov 14, 2018

@jreback jreback added this to the 0.24.0 milestone Nov 14, 2018

@jreback jreback added the Reshaping label Nov 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.