You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Due to a recent change in pandas, the read_hdf function closes HDF5 stores if it fails to read from the file. In my opinion, this is the expected behavior if read_hdf opened the file itself, but shouldn't happen if it was passed a file that is already open. One use case for this is a series of try/except blocks that try to read a series of different keys from an HDF5 file that may or may not have the individual keys, as in the example above. The attempt to read key2 will fail because the first attempt will have closed the file.
This issue was introduced in pull #25863 to address issue #25766 where read_hdf wasn't closing files that it hadn't opened itself.
The fix is pretty easy, just add if not isinstance(path_or_buf, HDFStore): before store.close() at the end of read_hdf().
Expected Output
Should not raise an Exception when reading key2.
Output of pd.show_versions()
[paste the output of pd.show_versions() here below this line]
INSTALLED VERSIONS
commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Linux
OS-release : 3.10.0-957.27.2.el7.x86_64
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : en_US.utf8
LANG : en_US.utf8
LOCALE : en_US.UTF-8
Code Sample, a copy-pastable example if possible
Problem description
Due to a recent change in pandas, the read_hdf function closes HDF5 stores if it fails to read from the file. In my opinion, this is the expected behavior if read_hdf opened the file itself, but shouldn't happen if it was passed a file that is already open. One use case for this is a series of try/except blocks that try to read a series of different keys from an HDF5 file that may or may not have the individual keys, as in the example above. The attempt to read key2 will fail because the first attempt will have closed the file.
This issue was introduced in pull #25863 to address issue #25766 where read_hdf wasn't closing files that it hadn't opened itself.
The fix is pretty easy, just add
if not isinstance(path_or_buf, HDFStore):
beforestore.close()
at the end of read_hdf().Expected Output
Should not raise an Exception when reading key2.
Output of
pd.show_versions()
[paste the output of
pd.show_versions()
here below this line]INSTALLED VERSIONS
commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Linux
OS-release : 3.10.0-957.27.2.el7.x86_64
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : en_US.utf8
LANG : en_US.utf8
LOCALE : en_US.UTF-8
pandas : 0.25.1
numpy : 1.17.2
pytz : 2019.2
dateutil : 2.8.0
pip : 19.2.3
setuptools : 41.0.1
Cython : 0.29.13
pytest : 5.1.2
hypothesis : None
sphinx : 2.2.0
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.1
IPython : 7.8.0
pandas_datareader: None
bs4 : 4.8.0
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.1
numexpr : 2.7.0
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.3.1
sqlalchemy : None
tables : 3.5.2
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
The text was updated successfully, but these errors were encountered: