Skip to content

BUG: Inconsistent returned objects when applying groupby aggregations #61503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
2 of 3 tasks
sylvainmouretfico opened this issue May 27, 2025 · 2 comments · Fixed by #61508
Closed
2 of 3 tasks

BUG: Inconsistent returned objects when applying groupby aggregations #61503

sylvainmouretfico opened this issue May 27, 2025 · 2 comments · Fixed by #61508
Labels
Apply Apply, Aggregate, Transform, Map Bug

Comments

@sylvainmouretfico
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
df = pd.DataFrame(columns=['Group', 'Data'])
df.groupby(['Group'], as_index=False)['Data'].agg('sum')
# Returns:
# Empty DataFrame
# Columns: [Group, Data]
# Index: []
def mysum(x):
  return sum(x)
df.groupby(['Group'], as_index=False)['Data'].agg(mysum)
# Returns:
# Series([], Name: Data, dtype: object)

Issue Description

When performing groupby aggregations on empty dataframe (with labeled columns), the outcome differs whether we use an internal aggregator or a custom function.
The difference of behaviour is problematic because when using internal aggregators (like 'sum'), the returned object is a dataframe with proper columns that we can select. However with custom functions, the returned object with an empty Series from which we cannot select columns.

This forces developers in this situation to check emptiness of the dataframe first.
This is not desirable from code conciseness point of view, but more importantly, we easily forget to check it and can therefore lead to errors.

Expected Behavior

Both approaches to apply groupby aggregations should return the same object, preferably a dataframe form which we can select columns.

Installed Versions

INSTALLED VERSIONS

commit : 0691c5c
python : 3.12.10
python-bits : 64
OS : Windows
OS-release : 11
Version : 10.0.26100
machine : AMD64
processor : Intel64 Family 6 Model 170 Stepping 4, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 2.2.3
numpy : 1.26.4
pytz : 2024.2
dateutil : 2.9.0.post0
pip : 25.1.1
Cython : None
sphinx : None
IPython : 9.2.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.6
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : 18.1.0
pyreadstat : None
pytest : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : 0.23.0
tzdata : 2024.2
qtpy : None
pyqt5 : None
@sylvainmouretfico sylvainmouretfico added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 27, 2025
@sylvainmouretfico sylvainmouretfico changed the title BUG: BUG: Inconsistent returned objects when applying groupby aggregations May 27, 2025
@sylvainmouretfico
Copy link
Author

sylvainmouretfico commented May 27, 2025 •
edited
Loading

I have explored the code a bit, and from what I can see, it would look like it would be required to update the following code in pandas/core/groupby/generic.py/SeriesGroupBy/aggregate:

        if self.ngroups == 0:
            # e.g. test_evaluate_with_empty_groups without any groups to
            #  iterate over, we have no output on which to do dtype
            #  inference. We default to using the existing dtype.
            #  xref GH#51445
            obj = self._obj_with_exclusions
            return self.obj._constructor(
                [],
                name=self.obj.name,
                index=self._grouper.result_index,
                dtype=obj.dtype,
            )

with the following:

        if self.ngroups == 0:
            # e.g. test_evaluate_with_empty_groups without any groups to
            #  iterate over, we have no output on which to do dtype
            #  inference. We default to using the existing dtype.
            #  xref GH#51445
            obj = self._obj_with_exclusions
            return self._wrap_aggregated_output(
                self.obj._constructor(
                    [],
                    name=self.obj.name,
                    index=self._grouper.result_index,
                    dtype=obj.dtype,
                )
            )

@arthurlw
Copy link
Member

Confirmed on main. PRs and investigations are welcome.

Thanks for raising this!

@arthurlw arthurlw added Apply Apply, Aggregate, Transform, Map and removed Needs Triage Issue that has not been reviewed by a pandas team member labels May 28, 2025
@arthurlw arthurlw mentioned this issue May 28, 2025
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Apply Apply, Aggregate, Transform, Map Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

BUG: Fix inconsistent returned objects when applying groupby aggregations arthurlw/pandas
2 participants
@sylvainmouretfico @arthurlw