Regression in 0.20: IO functions can't handle pathlib.Path objects anymore #16291

Closed
mruffalo opened this Issue May 8, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@mruffalo

mruffalo commented May 8, 2017

Code Sample, a copy-pastable example if possible

import numpy as np
import pandas as pd
from pathlib import Path
d = pd.DataFrame(np.random.randn(25).reshape((5, 5)))
d.to_pickle('test.pickle')
d2 = pd.read_pickle(Path('test.pickle'))

Problem description

In 0.19, Pandas' IO functions accepted PathLike objects as per PEP 519. This is broken in 0.20.1.

Expected Output

nothing, code should run without errors or output

Actual Output

Traceback (most recent call last):
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 83, in try_read
    return read_wrapper(lambda f: pkl.load(f))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 68, in read_wrapper
    return func(f)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 83, in <lambda>
    return read_wrapper(lambda f: pkl.load(f))
TypeError: file must have 'read' and 'readline' attributes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 88, in try_read
    lambda f: pc.load(f, encoding=encoding, compat=False))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 68, in read_wrapper
    return func(f)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 88, in <lambda>
    lambda f: pc.load(f, encoding=encoding, compat=False))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/compat/pickle_compat.py", line 187, in load
    fh.seek(0)
AttributeError: 'PosixPath' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 94, in read_pickle
    return try_read(path)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 92, in try_read
    lambda f: pc.load(f, encoding=encoding, compat=True))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 68, in read_wrapper
    return func(f)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 92, in <lambda>
    lambda f: pc.load(f, encoding=encoding, compat=True))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/compat/pickle_compat.py", line 187, in load
    fh.seek(0)
AttributeError: 'PosixPath' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 83, in try_read
    return read_wrapper(lambda f: pkl.load(f))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 68, in read_wrapper
    return func(f)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 83, in <lambda>
    return read_wrapper(lambda f: pkl.load(f))
TypeError: file must have 'read' and 'readline' attributes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 88, in try_read
    lambda f: pc.load(f, encoding=encoding, compat=False))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 68, in read_wrapper
    return func(f)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 88, in <lambda>
    lambda f: pc.load(f, encoding=encoding, compat=False))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/compat/pickle_compat.py", line 187, in load
    fh.seek(0)
AttributeError: 'PosixPath' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 97, in read_pickle
    return try_read(path, encoding='latin1')
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 92, in try_read
    lambda f: pc.load(f, encoding=encoding, compat=True))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 68, in read_wrapper
    return func(f)
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/io/pickle.py", line 92, in <lambda>
    lambda f: pc.load(f, encoding=encoding, compat=True))
  File "/home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/pandas/compat/pickle_compat.py", line 187, in load
    fh.seek(0)
AttributeError: 'PosixPath' object has no attribute 'seek'

Output of pd.show_versions()

>>> pd.show_versions() /home/mruffalo/.opt/python3.6-venv/lib/python3.6/site-packages/xarray/core/formatting.py:16: FutureWarning: The pandas.tslib module is deprecated and will be removed in a future version. from pandas.tslib import OutOfBoundsDatetime

INSTALLED VERSIONS

commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.8.0-51-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.20.1
pytest: None
pip: 9.0.1
setuptools: 35.0.2
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
xarray: 0.9.1
IPython: 6.0.0
sphinx: None
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.3.0
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None

@chris-b1

This comment has been minimized.

Show comment
Hide comment
@chris-b1

chris-b1 May 8, 2017

Contributor

Thanks for the report - does still seem to work with some io functions (tried read_csv).

Contributor

chris-b1 commented May 8, 2017

Thanks for the report - does still seem to work with some io functions (tried read_csv).

@chris-b1 chris-b1 added this to the 0.20.2 milestone May 8, 2017

@michaelaye

This comment has been minimized.

Show comment
Hide comment
@michaelaye

michaelaye May 8, 2017

Contributor

to_csv is broken as well, though.

Contributor

michaelaye commented May 8, 2017

to_csv is broken as well, though.

@chris-b1 chris-b1 referenced this issue May 9, 2017

Merged

BUG: pathlib.Path in io #16292

2 of 4 tasks complete

@jreback jreback closed this in #16292 May 12, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment