CLN: Remove `counts` Argument from Cython GroupBy Aggregations

[WillAyd]

All of the Cython aggregation functions accept counts as their second argument. In theory, it looks as if this parameter could assist in the handling of NaN values within a group, as dictated by the below lines of code:

pandas/pandas/core/groupby.py

Line 2398 in e022d9a

self._filter_empty_groups and not counts.all():

However, self._filter_empty_groups from what I can tell is always False. The only place this really gets exposed is in the BinGrouper constructor, but that is an internal class and I don't see any invocations of it that override the default value, essentially making the code linked above a noop.

I'm assuming that is part of an abandoned implementation, and if that's the case there's a decent amount of code cleanup / simplification that can be had between groupby.py and groupby_helpers.pxi.in if we remove counts

[WillAyd]

I was wrong on my original comment about self._filter_empty_groups always being False - it is set to True in the BaseGrouper initialization if there is more than one grouping.

That said, I think there's still something strange with the implementation here. The counts parameter is initialized to an array matching the ngroups parameter, which appears to exclude NaN values:

pandas/pandas/core/groupby.py

Line 2378 in e022d9a

counts = np.zeros(self.ngroups, dtype=np.int64)

In essence the, the counts variable local to the _cython_operation function will only give you the number of times that each non-NaN group label appears. Assuming that to be true, then the conditional in my original comment still doesn't serve a purpose because it has no notion of when NaN actually does occur

I noticed that #3729 is open and may impact how we go about this. Perhaps even some of this code was put here with the forward-looking intention of servicing that request? I would still lean towards removing the variable until that implementation actually happens to minimize the code-base. Curious if others have thoughts

[jreback]

there is an old impl that gets u nan groupings (ref from #3729) may want to have a look

[jbrockmendel]

@WillAyd did you ever look into this? Worth looking into as part of the current groupby-Exception push?

[WillAyd]

Haven't looked in quite some time, but I think if it simplifies things would make sense to remove

[jbrockmendel]

Looking at this, this seems to boil down to determining whether the if kind == "aggregate" and self._filter_empty_groups and not counts.all(): condition is ever satisfied. it isn't in the tests that make it to codecov: https://codecov.io/gh/pandas-dev/pandas/src/master/pandas/core/groupby/ops.py#L547, but maybe in e.g. slow tests?

[rhshadrach]

counts is now used to implement min_count:

pandas/pandas/core/groupby/ops.py

Lines 492 to 500 in 4fcee0e

	cutoff = max(0 if self.how in ["sum", "prod"] else 1, min_count)
	empty_groups = counts < cutoff
	if empty_groups.any():
	if result_mask is not None:
	assert result_mask[empty_groups].all()
	else:
	# Note: this conversion could be lossy, see GH#40767
	result = result.astype("float64")
	result[empty_groups] = np.nan

With that, can this be closed @WillAyd?