BUG: groupby.apply when func always returns None returns empty dataframe

[mvashishtha]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

print(pd.DataFrame(['a']).groupby(0).apply(lambda x: None))

Issue Description

Returns an empty dataframe with no columns or index

Expected Behavior

Should return something like

0
a   None
dtype: object

Installed Versions

INSTALLED VERSIONS
------------------
commit                : bdc79c146c2e32f2cab629be240f01658cfb6cc2
python                : 3.9.18.final.0
python-bits           : 64
OS                    : Darwin
OS-release            : 23.3.0
Version               : Darwin Kernel Version 23.3.0: Wed Dec 20 21:31:00 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6020
machine               : arm64
processor             : arm
byteorder             : little
LC_ALL                : None
LANG                  : en_US.UTF-8
LOCALE                : en_US.UTF-8

pandas                : 2.2.1
numpy                 : 1.26.3
pytz                  : 2023.3.post1
dateutil              : 2.8.2
setuptools            : 68.2.2
pip                   : 23.3.1
Cython                : None
pytest                : None
hypothesis            : None
sphinx                : None
blosc                 : None
feather               : None
xlsxwriter            : None
lxml.etree            : None
html5lib              : None
pymysql               : None
psycopg2              : None
jinja2                : None
IPython               : 8.18.1
pandas_datareader     : None
adbc-driver-postgresql: None
adbc-driver-sqlite    : None
bs4                   : None
bottleneck            : None
dataframe-api-compat  : None
fastparquet           : None
fsspec                : None
gcsfs                 : None
matplotlib            : None
numba                 : None
numexpr               : None
odfpy                 : None
openpyxl              : None
pandas_gbq            : None
pyarrow               : None
pyreadstat            : None
python-calamine       : None
pyxlsb                : None
s3fs                  : None
scipy                 : None
sqlalchemy            : None
tables                : None
tabulate              : None
xarray                : None
xlrd                  : None
zstandard             : None
tzdata                : 2023.4
qtpy                  : None
pyqt5                 : None

[Delengowski]

Was this ever the case? Admittedly I've never done something like this so I just don't know. Been a user since 0.23, I can't think of a time I've used a function like this. Docs are unclear if its special cased or not.

[rhshadrach]

Agreed this should be added to the documentation in the Notes section of apply. This is to allow certain groups to be filtered.

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
gb = df.groupby("a")

def foo(x):
    if x.iloc[0, 0] == 5:
        return None
    else:
        return x**2

print(gb.apply(foo, include_groups=False))
#       b
# a      
# 1 0  16
# 3 2  36

Still - in the OP case we should return a DataFrame with the same columns and dtypes as the input. So labeling this both doc and bug.

[dontgoto]

take