Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misleading error for pd.read_msgpack #27160

smspillaz opened this issue Jul 1, 2019 · 5 comments


Copy link

commented Jul 1, 2019

Code Sample, a copy-pastable example if possible

import pandas as pd

Problem description

Such an error is misleading because it suggests that there is a problem with the datatype being passed, not that the path does not exist. The error raised is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".local/anaconda3/lib/python3.7/site-packages/pandas/io/", line 226, in read_msgpack
    raise ValueError('path_or_buf needs to be a string file path or file-like')
ValueError: path_or_buf needs to be a string file path or file-like

Expected Output

Raise an error indicating that the path was not found.

Output of pd.show_versions()

>>> pd.show_versions()


commit: None
python-bits: 64
OS: Linux
OS-release: 4.18.0-24-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None

pandas: 0.24.2
pytest: 4.3.1
pip: 19.0.3
setuptools: 40.8.0
Cython: 0.29.6
numpy: 1.16.2
scipy: 1.2.1
pyarrow: None
xarray: None
IPython: 7.4.0
sphinx: 1.8.5
patsy: 0.5.1
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: 1.2.1
tables: 3.5.1
numexpr: 2.6.9
feather: None
matplotlib: 3.0.3
openpyxl: 2.6.1
xlrd: 1.2.0
xlwt: 1.3.0
xlsxwriter: 1.1.5
lxml.etree: 4.3.2
bs4: 4.7.1
html5lib: 1.0.1
sqlalchemy: 1.3.1
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None


This comment has been minimized.

Copy link

commented Jul 1, 2019


This comment has been minimized.

Copy link

commented Jul 1, 2019

yeah this is true of several routines (e.g. read_json), there is an issue about this somewhere. but for msgpack since we are deprecated, this is out of scope (would take a reasonable patch though).

@jreback jreback closed this Jul 1, 2019

@jreback jreback modified the milestones: No action, 0.25.0 Jul 1, 2019

@jreback jreback reopened this Jul 1, 2019


This comment has been minimized.

Copy link

commented Jul 1, 2019

as PR is submitted :->


This comment has been minimized.

Copy link

commented Jul 1, 2019

I think the first argument of read_msgpack can also be data.

I think that assuming a string passed to pd.read_msgpack is a filepath and then raising if not found is OK?

the data as bytes works as intended.

the docs for pandas.DataFrame.to_msgpack are misleading suggest that a string is returned when bytes are returned...

path : string File path, buffer-like, or None
if None, return generated string
>>> import numpy as np
>>> import pandas as pd
>>> from pandas import DataFrame
>>> df = DataFrame(np.random.randn(10, 2))
>>> df.to_msgpack(None)
>>> pd.read_msgpack(df.to_msgpack(None))
sys:1: FutureWarning: The read_msgpack is deprecated and will be removed in a future version.
It is recommended to use pyarrow for on-the-wire transmission of pandas objects.
          0         1
0  0.257572  0.284149
1 -1.374214  2.427524
2 -0.166749 -0.141252
3  0.105612  0.909719
4  0.535428 -0.260687
5 -0.062252  1.308856
6 -0.937890  2.729950
7 -0.408451 -1.030632
8  1.986504  0.854142
9  0.533630  0.823308

@jreback jreback modified the milestones: 0.25.0, Contributions Welcome Jul 2, 2019

U09Kane added a commit to U09Kane/pandas that referenced this issue Jul 3, 2019

@U09Kane U09Kane referenced this issue Jul 3, 2019


Misleading error for pd.read_msgpack #27201

4 of 4 tasks complete

@jreback jreback modified the milestones: Contributions Welcome, 0.25.0 Jul 6, 2019


This comment has been minimized.

Copy link

commented Jul 19, 2019


I'm confused by the deprecation msg. Compare to read_msgpack, which function in pyarrow is the replacement?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
5 participants
You can’t perform that action at this time.