[Python] Add function to "flatten" structs within tables

[asfimport]

See discussion in #17866

When a user has a struct column, it may be more efficient to flatten the struct into multiple columns of the form struct_name.field_name for each field in the struct. Then when you call to_pandas, Python dictionaries do not have to be created, and the conversion will be much more efficient

Reporter: Wes McKinney / @wesm
Assignee: Antoine Pitrou / @pitrou

Related issues:

• [C++/Python] Add method to flatten a struct array (Dependent)
• [Python] Segmentation fault when loading total 2GB of parquet files (is related to)

PRs and other links:

• GitHub Pull Request #1768

_{Note: This issue was originally created as ARROW-1886. Please see the migration documentation for further details.}

[asfimport]

Li Jin / @icexelloss:
Does it make sense to have a mode for table.to_pandas to turn arrow struct into multi index in pandas?

[asfimport]

Wes McKinney / @wesm:
Probably not – we only create a MultiIndex when we have the appropriate metadata for this. We could use the same flattening tool to convert a struct into a MultiIndex though

[asfimport]

Antoine Pitrou / @pitrou:
Should this happen on the C++ side as well?

[asfimport]

Wes McKinney / @wesm:
I believe so, yes

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 1768
#1768