Skip to content

reindex fails on Dataset from MultiIndex DataFrame with RuntimeError #10347

Closed
@csernazs

Description

@csernazs
Contributor

What happened?

The reproducer code raises
RuntimeError: Cannot set name on a level of a MultiIndex. Use 'MultiIndex.set_names' instead.
from the last statement.

Traceback:

Traceback (most recent call last):
  File "/tmp/xarray-bug/repro.py", line 53, in <module>
    data2b = data1.reindex(y=base2.y)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/xarray-bug/.venv/lib/python3.12/site-packages/xarray/core/dataset.py", line 3571, in reindex
    return alignment.reindex(
           ^^^^^^^^^^^^^^^^^^
  File "/tmp/xarray-bug/.venv/lib/python3.12/site-packages/xarray/structure/alignment.py", line 974, in reindex
    aligner = Aligner(
              ^^^^^^^^
  File "/tmp/xarray-bug/.venv/lib/python3.12/site-packages/xarray/structure/alignment.py", line 172, in __init__
    self.indexes, self.index_vars = self._normalize_indexes(indexes)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/xarray-bug/.venv/lib/python3.12/site-packages/xarray/structure/alignment.py", line 209, in _normalize_indexes
    pd_idx.name = k
    ^^^^^^^^^^^
  File "/tmp/xarray-bug/.venv/lib/python3.12/site-packages/pandas/core/indexes/base.py", line 1690, in name
    raise RuntimeError(
RuntimeError: Cannot set name on a level of a MultiIndex. Use 'MultiIndex.set_names' instead.

This happens with the latest xarray (2025.4.0). I could bisect the version for this issue, and the last version where it runs fine is 2024.5.0 (and the next version, which is 2024.6.0 breaks it).

Thanks,
Zsolt

What did you expect to happen?

Code run without errors.

Minimal Complete Verifiable Example

import xarray as xr
import pandas as pd
import numpy as np


# Generate two arrays of random numbers
rand1 = np.random.randn(15)
rand2 = np.random.randn(12)

# Create a 5x3 DataArray from rand1 with labeled dimensions and convert it to a Dataset
data1 = xr.DataArray(
    rand1.reshape((5, 3)),
    dims=("x", "y"),
    coords={"x": [1, 2, 3, 4, 5], "y": [1, 2, 3]},
).to_dataset(name="value")

# Create a 3x4 DataArray from rand2 with labeled dimensions and convert it to a Dataset
base1 = xr.DataArray(
    rand2.reshape(3, 4),
    dims=("x", "y"),
    coords={"x": [100, 200, 300], "y": [1, 2, 3, 4]},
).to_dataset(name="base_value")

# Create an equivalent Dataset to data1 using a pandas MultiIndex DataFrame
data2 = xr.Dataset.from_dataframe(
    pd.DataFrame(
        rand1,
        columns=["value"],
        index=pd.MultiIndex.from_product([[1, 2, 3, 4, 5], [1, 2, 3]], names=["x", "y"]),
    )
)

# Create an equivalent Dataset to base1 using a pandas MultiIndex DataFrame
base2 = xr.Dataset.from_dataframe(
    pd.DataFrame(
        rand2,
        columns=["base_value"],
        index=pd.MultiIndex.from_product([[100, 200, 300], [1, 2, 3, 4]], names=["x", "y"]),
    )
)

# Verify that the two ways of constructing the datasets yield the same results
# These succeed
assert data1.equals(data2)
assert base1.equals(base2)

# Reindex `data1` to align its 'y' dimension with the 'y' coordinates of `base1`
data1b = data1.reindex(y=base1.y)  # succeeds

# Reindex `data1` again, but now aligning with `base2` (same structure as `base1`)
# fails with
# RuntimeError: Cannot set name on a level of a MultiIndex. Use 'MultiIndex.set_names' instead.
data2b = data1.reindex(y=base2.y)

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
    Complete example — the example is self-contained, including all data and the text of any traceback.
    Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
    New issue — a search of GitHub Issues suggests this is not a duplicate.
    Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

Anything else we need to know?

No response

Environment

numpy==2.2.6
packaging==25.0
pandas==2.2.3
python-dateutil==2.9.0.post0
pytz==2025.2
six==1.17.0
tzdata==2025.2
xarray==2025.4.0

I'm using Linux on x86-64 with python 3.12.

Activity

added
needs triageIssue that has not been reviewed by xarray team member
on May 23, 2025
benbovy

benbovy commented on May 27, 2025

@benbovy
Member

Hmm I don't see any specific change on the Xarray side that would have caused this issue.

However, the pandas behavior is not exactly the one that I would have expected:

>>> import pandas as pd
>>> pd.__version__
'2.2.3'
>>> midx = pd.MultiIndex.from_product([[100, 200, 300], [1, 2, 3, 4]], names=["x", "y"])
>>> y_idx = midx.levels[1]
>>> y_idx
Index([1, 2, 3, 4], dtype='int64', name='y')
>>> y_idx.name = "y"
RuntimeError: Cannot set name on a level of a MultiIndex. Use 'MultiIndex.set_names' instead.

It looks like midx.levels[1] does not return a copy but a view of the MultiIndex. I suspect it wasn't the case before and perhaps the change in behavior is related to pandas's copy on write migration?

Changing the line

pd_idx.name = k
to

if pd_idx.name != k:
    pd_idx = pd_idx.copy()
    pd_idx.name = k

should fix the issue.

added and removed
needs triageIssue that has not been reviewed by xarray team member
on May 27, 2025
csernazs

csernazs commented on May 29, 2025

@csernazs
ContributorAuthor

hi @benbovy ,

Thanks for the fix. I confirm that the fix works.

Are you going to fix this in xarray (do you want the fix be merged)?

Looking into pandas side, they also provide set_names() method which works like this:

if pd_idx.name != k:
    pd_idx = pd_idx.set_names(k)

(it returns a copy, similar to your fix)

Thanks again,
Zsolt

benbovy

benbovy commented on May 30, 2025

@benbovy
Member

pd_idx.set_names(k) won't work if pd_idx is a MultiIndex. Even for the latter, we set the pd_idx.name attribute such that it corresponds to the dimension of the Xarray coordinate variables created for the multi-index levels.

Are you going to fix this in xarray (do you want the fix be merged)?

Yes it would be nice to fix this in Xarray. Are you willing to submit a PR? I'm happy to review it.

csernazs

csernazs commented on May 30, 2025

@csernazs
ContributorAuthor

Yes, I'm willing to submit the PR soon.

csernazs

csernazs commented on May 30, 2025

@csernazs
ContributorAuthor

I've submitted a PR (#10381) for this.
Note that this is my first PR here, so if you spot anything wrong with it, I'm happy to receive comments / feedback.

Thank you,
Zsolt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @csernazs@benbovy

      Issue actions

        reindex fails on Dataset from MultiIndex DataFrame with RuntimeError · Issue #10347 · pydata/xarray