Overflow problem when writing large dataframe to HDF5 with table format.

[kid3night]

Code Sample, a copy-pastable example if possible

# Your code here
self.feature_concated.to_hdf(self.h5_result_name, key='feature_data', mode='a', format='table')
self.feature_concated.to_hdf(self.h5_result_name, key='feature_data', mode='a')

Problem description

I need to save a large DataFrame to Hdf5 file so I use the code above to save the DataFrame. And a RuntimeWarning will show up:

F:\Anaconda3\lib\site-packages\tables\leaf.py:357: RuntimeWarning: overflow encountered in long_scalars
expected_mb = (expectedrows * rowsize) // MB

But when I don't use table format hdf to save it, it will not have such problem and everything goes well.
I'm wondering what the cause of this problem is and how could I solve it?

Expected Output

F:\Anaconda3\lib\site-packages\tables\leaf.py:357: RuntimeWarning: overflow encountered in long_scalars
expected_mb = (expectedrows * rowsize) // MB

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]
commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Windows
OS-release: 2012ServerR2
machine: AMD64
processor: Intel64 Family 6 Model 63 Stepping 2, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.3
pytest: 3.6.3
pip: 18.0
setuptools: 39.2.0
Cython: 0.28.3
numpy: 1.14.5
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 6.4.0
sphinx: 1.7.5
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: 1.2.1
tables: 3.4.4
numexpr: 2.6.5
feather: None
matplotlib: 2.2.2
openpyxl: 2.5.4
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.5
lxml: 4.2.2
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.2.8
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[WillAyd]

The warning you are getting is coming from PyTables and not pandas; not sure there is a lot that can be done here but if anything should raise the question with that project

[kid3night]

It's about pandas. Although the warning is about pytables but just as I noted, it's when I use to_hdf using format='table' that the issue occurred.