make_meta over a Dask Dataframe returns a reference, not a new object

[albarji]

Describe the issue:

Reading the documentation of make_meta it states that

This method creates meta-data based on the type of x

so my understanding is that a new object is returned. However, one can check that when running make_meta over a Dask Dataframe a reference to the dataframe meta is returned. Thus, if any change is made to the returned meta, the meta of the Dataframe is modified as well.

Minimal Complete Verifiable Example:

import dask.dataframe as dd
import pandas as pd

df = pd.DataFrame({
    "col1": [1, 2, 3],
    "col2": [4, 5, 6],
})
df = dd.from_pandas(df, npartitions=2)

print(df.columns)  # returns Index(['col1', 'col2'], dtype='object')

from dask.dataframe.utils import make_meta
meta = make_meta(df)
meta["flag"] = pd.Series([], dtype="bool")
print(df.columns)  # returns Index(['col1', 'col2', 'flag'], dtype='object')

Anything else we need to know?:

In my experience make_meta is very useful to obtain the current meta of a Dataframe and then update it with the necessary changes to provide appropriate meta information to methods such as map_partitions or assign, so that Dask knows how you intend to change the structure of the Dataframe. But since make_meta returns a reference it seems we are forced to make changes to a copy of this meta object, which is inconvenient. Is there any design reason for returning a reference instead of a copy?

Environment:

• Dask version: 2024.1.0
• Python version: 3.12.1
• Operating System: Ubuntu 22.04.3 LTS
• Install method (conda, pip, source): pip