BUG: dataframe.groupby raises type error when grouping with an object type

[Guilouf]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
from operator import attrgetter


class Du:

    def __init__(self, x):
        self.value = x

    def __hash__(self):
        return hash(self.value)

    def __eq__(self, other):
        return self.value == other.value

# Indexing with dicts
dict_example = {
    Du('X'): [1, 2],
    Du(5): [3, 4],
}

print(dict_example[Du('X')])  # Du() instances works for dict indexing


# Grouping with dataframes
data = {
    'Money': [1000, 2500, 1500, 1200, 3000],
    'Dummy': [Du(1), Du('Y'), Du('X'), Du(5), Du(5)]
}
df = pd.DataFrame(data)

# create a column with a value of each Du object 'value' attribute
df['dum_value'] = df['Dummy'].apply(attrgetter('value'))


val = df.mean(numeric_only=True)
print(val)

grouped = df.groupby('dum_value')  # works even if 'X' < 5 is not possible
print(grouped)

val2 = grouped.mean(numeric_only=True)
print(val2)

grouped_object = df.groupby('Dummy')

val3 = grouped_object.mean(numeric_only=True)  # raises TypeError when grouped by 'Du' objects
print(val3)

Issue Description

When comptuting means with groupby, if the groupby is performed on an object, then it raises a type error, TypeError: '<' not supported between instances ....

Expected Behavior

however, when performing a groupby on another column, even with type that cannot be numerically compared (e.g 'X' < 5), groupby works.
I think for a custom object, the behaviour should be the same as dict, i.e comparing hashes and equality.

Installed Versions

INSTALLED VERSIONS

commit : 4bfe3d0
python : 3.8.13.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-109-generic
Version : #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.4.2
numpy : 1.22.3
pytz : 2020.1
dateutil : 2.8.1
pip : 22.0.4
setuptools : 57.5.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.8.0
html5lib : None
pymysql : None
psycopg2 : 2.8.5
jinja2 : None
IPython : 7.28.0
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
markupsafe : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : 3.0.6
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : 2.0.1
xlwt : 1.3.0
zstandard : None

[Guilouf]

i found a workaround, with grouped_object = df.groupby('Dummy', sort=False), but still, why is it working with the column dum_value, which appear to have values that are non sortable (int and chars) ?

[simonjayhawkins]

Thanks @Guilouf for the report.

As you discovered, the default for .groupby() is sort=True, and therefore raises if the objects can't be sorted.

This default is a long standing behavior, so unlikely to change, although as stated in the docs "Get better performance by turning this off." so IMO makes sense for the default to be False for both robustness and performance.

why is it working with the column dum_value, which appear to have values that are non sortable (int and chars) ?

pandas/pandas/core/algorithms.py

Lines 1767 to 1772 in f9810c7

	def _sort_mixed(values) -> np.ndarray:
	"""order ints before strings in 1d arrays, safe in py3"""
	str_pos = np.array([isinstance(x, str) for x in values], dtype=bool)
	nums = np.sort(values[~str_pos])
	strs = np.sort(values[str_pos])
	return np.concatenate([nums, np.asarray(strs, dtype=object)])

[mroeschke]

Agreed this is dictated by the sort keyword than can be toggled by the user. The default here may change to sort=None in the future, but that topic is orthogonal to the immediate issue so closing.