BUG: read_pickle and period data #10439

Closed
scarrucciu opened this Issue Jun 25, 2015 · 1 comment

Comments

Projects
None yet
2 participants

I can pickle dataframes with periods but after import Period format becomes corrupted. After pickle Period stride appears to be none instead of 'Q' in the following example.


import pandas as pd

df = pd.DataFrame({'period': [pd.Period('2014Q4')] * 5})
msk = df['period'] == pd.Period('2014Q4')
print msk

df.to_pickle('tmp.pkl')
df = pd.read_pickle('tmp.pkl')

msk = df['period'] == pd.Period('2014Q4')

Traceback (most recent call last):
  File "/Users/scarrucciu/Downloads/period_pickle_test.py", line 11, in 
    msk = df['period'] == pd.Period('2014Q4')
  File "/Users/scarrucciu/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 614, in wrapper
    res = na_op(values, other)
  File "/Users/scarrucciu/anaconda/lib/python2.7/site-packages/pandas/core/ops.py", line 562, in na_op
    result = lib.scalar_compare(x, y, op)
  File "pandas/lib.pyx", line 696, in pandas.lib.scalar_compare (pandas/lib.c:12434)
  File "das/src/period.pyx", line 743, in pandas._period.Period.__richcmp__ (pandas/src/period.c:11121)
ValueError: Cannot compare non-conforming periods

INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Darwin
OS-release: 13.4.0
machine: x86_64
processor: i386
byteorder: little
pandas: 0.16.2

Contributor

jreback commented Jun 25, 2015

ATM, Period doesn't have ANY pickle support so not really sure what it is doing.

pull-requests are welcome! (basically need to add __reduce__/__setstate__), see for example pandas/tslib.pyx/Timestamp on how to do it.

Also not tested. PeriodIndex has support for this, but I don't think tested either.

jreback added this to the 0.17.0 milestone Jun 25, 2015

@jreback jreback added a commit to jreback/pandas that referenced this issue Aug 20, 2015

@jreback Spencer Carrucciu + jreback ENH: pickle support for Period #10439 0525684

jreback closed this in #10866 Aug 20, 2015

@springcoil springcoil added a commit to springcoil/pandas that referenced this issue Aug 21, 2015

@springcoil Spencer Carrucciu + springcoil ENH: pickle support for Period #10439
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs
7c41431

@springcoil springcoil added a commit to springcoil/pandas that referenced this issue Aug 21, 2015

@springcoil Spencer Carrucciu + springcoil ENH: pickle support for Period #10439
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

ENH: pickle support for Period #10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs
822df81

@springcoil springcoil added a commit to springcoil/pandas that referenced this issue Aug 22, 2015

@springcoil Spencer Carrucciu + springcoil ENH: pickle support for Period #10439
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs
3072e48

@springcoil springcoil added a commit to springcoil/pandas that referenced this issue Aug 22, 2015

@springcoil Spencer Carrucciu + springcoil ENH: pickle support for Period #10439
update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

ENH: pickle support for Period #10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs
535fbd2

@springcoil springcoil added a commit to springcoil/pandas that referenced this issue Aug 30, 2015

@springcoil springcoil BUG: 10633 and 10800 fix
BUG: 10633 - some last errors removed

ENH: pickle support for Period #10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs

TST: Changes in test

ENH: pickle support for Period #10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

ENH: pickle support for Period #10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs

ERR: 10720

BUG: 10633 and 10800 fix

merging

ENH: pickle support for Period #10439

update legacy_storage for pickles

update pickles/msgpack for 0.16.2

Added tests for ABC Types, Issue #10828

TST: #10822, skip tests on windows for odd error message in to_datetime with unicode

COMPAT:Allow multi-indexes to be written to excel.

(Even though they cannot be read back in.)

Closes #10564

DOC: typo

A few changes in docs

TST: Changes in test

Fixing a slight messup

DOC:Updating consistent imports in the merging.rst file #9886

DOC: GH9886 Part V

DOC: GH9886 Part V - some merging issues
f2b1bbc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment