[SPARK-46931][PS] Implement {Frame, Series}.to_hdf

[zhengruifeng]

What changes were proposed in this pull request?

Implement {Frame, Series}.to_hdf

Why are the changes needed?

pandas parity

Does this PR introduce any user-facing change?

yes

In [3]: df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}, index=['a', 'b', 'c'])

In [4]: df.to_hdf('/tmp/data.h5', key='df', mode='w')

In [5]: psdf = ps.from_pandas(df)

In [6]: psdf.to_hdf('/tmp/data2.h5', key='df', mode='w')
/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/utils.py:1015: PandasAPIOnSparkAdviceWarning: `to_hdf` loads all data into the driver's memory. It should only be used if the resulting DataFrame is expected to be small.
  warnings.warn(message, PandasAPIOnSparkAdviceWarning)
                                                                                
In [7]: !ls /tmp/*h5
/tmp/data.h5    /tmp/data2.h5

In [8]: !ls -lh /tmp/*h5
-rw-r--r--@ 1 ruifeng.zheng  wheel   6.9K Jan 31 12:21 /tmp/data.h5
-rw-r--r--@ 1 ruifeng.zheng  wheel   6.9K Jan 31 12:21 /tmp/data2.h5

How was this patch tested?

manually test, hdf requires additional library pytables which in turn needs many prerequisites

since pytables is just a optional dep of Pandas, so I think we can avoid adding it to CI first.

Was this patch authored or co-authored using generative AI tooling?

no

[dongjoon-hyun]

+1, LGTM.

[dongjoon-hyun]

Thank you so much for adding this, @zhengruifeng and @HyukjinKwon .

[zhengruifeng]

thanks @dongjoon-hyun and @HyukjinKwon for reviews