BUG: Some str reducers fail in Groupby.transform with axis=1

[rhshadrach]

Reducers:

• corrwith
• first - BUG: .transform(...) with "first" and "last" fail when axis=1 #46074
• idxmax
• idxmin
• last - BUG: .transform(...) with "first" and "last" fail when axis=1 #46074
• ngroup - BUG: .transform(ngroup) for axis=1 grouper #50761
• nth

All raise a ValueError when used with axis=1, e.g.

df = DataFrame({"a": [1, 2], "b": [3, 4], "c": [5, 6]}, index=["x", "y"])
result = df.groupby([0, 0, 1], axis=1).transform("first")

[iofall]

For idxmax and idxmin.

When we make the transform call, result = df.groupby([0, 0, 1], axis=1).transform("idxmax"). The axis information from the groupby is not used.

transform calls _transform which then calls idxmax on this line:

pandas/pandas/core/groupby/groupby.py

Line 1712 in 4c5db2e

result = getattr(self, func)(*args, **kwargs)

In the idxmax function we expect axis as an argument, which is by default set to 0. So even if the axis for self is 1, the axis for idxmax will not get overridden since it is not passed in the args or kwargs in the above line.

pandas/pandas/core/groupby/generic.py

Lines 1775 to 1781 in 4c5db2e

	def idxmax(
	self,
	axis: Axis = 0,
	skipna: bool = True,
	numeric_only: bool = False,
	) -> DataFrame:
	axis = DataFrame._get_axis_number(axis)

This is why it works for axis = 0, since it will take 0 by default. But to work with axis = 1, we will have to explicitly pass it in the kwargs as .transform("idxmax", axis = 1)

So, to fix:

• One way could be to explicitly pass the required axis and mention that as a caveat in the documentation.
• The second one could be to modify the _transform method to check if axis exists in kwargs, if not add it as self.axis inside the else block.

Let me know which way makes the most sense and I will make a PR for it.

[iofall]

For ngroup, it seems the error is only raised when index length does not match the len of the list provided.

df = pd.DataFrame({"a": [1, 2, 3], "b": [3, 4, 5], "c": [5, 6, 7]}, index=["x", "y", "z"])
result = df.groupby([0, 0, 1], axis=1).transform("ngroup")

The above transform will work, but:

df = pd.DataFrame({"a": [1, 2,], "b": [3, 4], "c": [5, 6]}, index=["x", "y"])
result =  df.groupby([0, 0, 1], axis=1).transform("ngroup")

will fail for .ngroup() as well as .transform("ngroup")

---

For nth,

>>> result = df.groupby([0, 0, 1], axis=1).transform("nth", 1)

# Output
ValueError: 'nth' is not a valid function name for transform(name)

It fails because nth does not exist in base.transform_kernel_allowlist when we check for it here:

pandas/pandas/core/groupby/groupby.py

Lines 1693 to 1695 in 4c5db2e

	elif func not in base.transform_kernel_allowlist:
	msg = f"'{func}' is not a valid function name for transform(name)"
	raise ValueError(msg)

[rhshadrach]

Thanks for digging into these!

So, to fix:

• One way could be to explicitly pass the required axis and mention that as a caveat in the documentation.
• The second one could be to modify the _transform method to check if axis exists in kwargs, if not add it as self.axis inside the else block.

I don't think we should require the user to pass axis=1 twice to get axis=1 behavior. The 2nd looks like a good solution to me, assuming specification of axis gives the correct result (the column label of the min/max value).

It fails because nth does not exist in base.transform_kernel_allowlist when we check for it here:

This looks correct to me - nth is a filter and as such we may end up with multiple or no values in a given group. As there isn't a reliable way to transform, I believe raising is the correct thing to do.

[rhshadrach]

Closing since axis=1 is deprecated.