ENH: Support large string types in the interchange protocol

[stinodego]

Feature Type

• Adding new functionality to pandas

• Changing existing functionality in pandas

• Removing existing functionality in pandas

Problem Description

The interchange protocol supports the large string type. Pandas could also support it, but currently does not. It only supports 'small' strings.

Feature Description

This currently works:

import pandas as pd

df = pd.Series([], name="a", dtype="string[pyarrow]").to_frame()
dfi = df.__dataframe__()
result = pd.api.interchange.from_dataframe(dfi)
# result is equal to df

This does not:

import pandas as pd

df = pd.Series([], name="a", dtype="large_string[pyarrow]").to_frame()
dfi = df.__dataframe__()
result = pd.api.interchange.from_dataframe(dfi)
# ValueError: Data type large_string[pyarrow] not supported by interchange protocol

Alternative Solutions

n/a

Additional Context

@MarcoGorelli FYI. Supporting large string would mean better interop with Polars and pyarrow.

[phofl]

Yeah this got more urgent since we switched to large strings by default