[Python] Conversion of pandas.SparseArray columns in pandas.DataFrames to pyarrow.Table and back

[asfimport]

We would like to convert sparse columns from Pandas to Arrow:

import numpy as np
import pandas
import pyarrow

arr = pandas.Series([1, 2, 3])
sparr = pandas.SparseArray(np.array([1, 0, 0], dtype='int64'))
df = pandas.DataFrame({'sparr': sparr, 'arr': arr})

table = pyarrow.table(df)
df == table.to_pandas()

I assume pandas.SparseArray is a 1D sparse COO Tensor that would map to pyarrow.SparseTensorCOO.

Reporter: Rok Mihevc / @rok
Watchers: Rok Mihevc / @rok

_{Note: This issue was originally created as ARROW-6327. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
Unfortunately a Arrow sparse tensor cannot be a Table column. Suggest closing.