Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: groupby.agg raises KeyError: "['col1', 'col5'] not found in axis" #52760

Closed
2 of 3 tasks
anmyachev opened this issue Apr 18, 2023 · 1 comment · Fixed by #52850
Closed
2 of 3 tasks

BUG: groupby.agg raises KeyError: "['col1', 'col5'] not found in axis" #52760

anmyachev opened this issue Apr 18, 2023 · 1 comment · Fixed by #52850
Assignees
Labels
Apply Apply, Aggregate, Transform Bug Categorical Categorical Data Type Groupby Regression Functionality that used to work in a prior pandas version
Milestone

Comments

@anmyachev
Copy link
Contributor

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd

pandas_df = pd.DataFrame(
    {
        "col1": [0, 1, 2, 3],
        "col2": [4, 5, np.NaN, 7],
        "col3": [np.NaN, np.NaN, 12, 10],
        "col4": [17, 13, 16, 15],
        "col5": [-4, -5, -6, -7],
    }
)

pandas_df = pandas_df.astype({"col1": "category"})
pandas_df["col1"] = pandas_df["col1"].cat.as_ordered()
pandas_groupby = pandas_df.groupby(by=["col1", "col5"], as_index=False)
print(pandas_groupby["col3"].agg(["mean"]))  # <-- *** KeyError: "['col1', 'col5'] not found in axis" (in pandas==2.0.0)

Issue Description

The example works fine in pandas==1.5.3 but doesn't work in pandas==2.0.0.

Expected Behavior

I think the test should work for pandas 2.0.0.

Installed Versions

Replace this line with the output of pd.show_versions()

@anmyachev anmyachev added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 18, 2023
@rhshadrach rhshadrach self-assigned this Apr 18, 2023
@rhshadrach rhshadrach added Groupby Regression Functionality that used to work in a prior pandas version and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 18, 2023
@rhshadrach rhshadrach added this to the 2.0.1 milestone Apr 18, 2023
@rhshadrach
Copy link
Member

Thanks for the report! This is due to #50744 - we evaluate whether a grouping is "in-axis" or not when the groupby instance is first created, and don't reevaluate this upon column selection. We end up thinking a grouping is in-axis and try to drop it because of as_index=False.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform Bug Categorical Categorical Data Type Groupby Regression Functionality that used to work in a prior pandas version
Projects
None yet
2 participants