Create NestedPandas from a flat source table assuming some columns to be object columns

[hombit]

Feature request

It would be really nice if we could generate NestedFrame with both "base" and "nested" columns from a single flat data-frame.

NestedFrame.from_flat(df, base_columns=["a", "b"])

It would be super-great if user could specify base_columns=None and we just guess which columns are base columns.

Before submitting
Please check the following:

• I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
• I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
• If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.

[hombit]

It would be super-great if user could specify base_columns=None and we just guess which columns are base columns.

We decided to drop this part

[dougbrn]

@nevencaplar has a use case for this that we should build around, this operation is something that we should try to have this from_flat function cover:

import nested_pandas as npd

# Step 1: Read the Parquet file into a pandas DataFrame
file_path = '/epyc/users/ncaplar/data/drw_per_sim/sources/sim_drw_023922.87-000119.6.parquet'
nf = npd.read_parquet(file_path)

nf_base = nf[["lc_nr", "drw_amp", "drw_tau"]].drop_duplicates(subset="lc_nr").set_index("lc_nr")

nf_nested = nf[["lc_nr", "t", "y"]].set_index("lc_nr")

nf_base.add_nested(nf_nested, "nested")

Looking at what we have above, I think we could implement this such that the above workflow could be simplified to:

import nested_pandas as npd

# Step 1: Read the Parquet file into a pandas DataFrame
file_path = '/epyc/users/ncaplar/data/drw_per_sim/sources/sim_drw_023922.87-000119.6.parquet'
nf = npd.read_parquet(file_path).set_index("lc_nr")

npd.NestedFrame.from_flat(nf, base_columns=["drw_amp", "drw_tau"], nested_columns=["t","y"])

This use case requires a drop_duplicates on all base_columns, which I think is the operation we'd want for most use cases of this function, potentially we should provide additional kwargs from drop_duplicates to better control the output of the base columns?