DOC: when accessed from inside a column-wise df.apply(..., axis=1), row.index refers to df.columns, and row.name refers to df.index value

[smcinerney]

DOC BUG: clarification for the function/lambda called inside apply()
When accessed from inside a column-wise df.apply(lambda row:..., axis=1), things get counterintuitive:

• row is now a (horizontal) Series...
• row.index is now the column names from df.columns
• row.name is the (individual) row-index value of that row in the parent df

Code Sample, a copy-pastable example if possible

df = pd.DataFrame({'x':[1,2,3], 'y':[4,5,6]})
df.apply(lambda row: row.index, axis=1)
0    Index(['x', 'y'], dtype='object')
1    Index(['x', 'y'], dtype='object')
2    Index(['x', 'y'], dtype='object')

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.7.3.final.0
python-bits: 64
OS: Darwin
OS-release: 18.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.24.2

[MarcoGorelli]

row.index is now the column names from df.columns

The docs currently read

Objects passed to the function are Series objects whose index is either the DataFrame’s index (axis=0) or the DataFrame’s columns (axis=1).

Do you have a suggestion on how to word it more clearly?

[MarcoGorelli]

@simonjayhawkins - should the doc be reworded somehow, or is it alright as it is? (IMO it's clear already, it even includes examples for both axis=0 and axis=1)

EDIT

closing for now, feel free to reopen if you have any suggestions :)

[smcinerney]

I propose the following, you might be able to edit it to be somewhat more terse. But I think we do need to explicitly spell out the counterintuitive behavior of a row-wise apply (it took me ages to debug what was going on, and why my parent dataframe's index had disappeared within the apply):

Objects passed to the function are Series objects whose index depends on whether the parameter axis = 0 or 1:

 - axis=0 case (column-wise): the index passed is the same as the parent dataframe's index
 - axis=1 case (row-wise): the index passed is actually the parent dataframe's *columns*
  - This has the following counterintuitive implications within a row-wise `df.apply(lambda row:..., axis=1)`:
   - row is now a (horizontal) Series...
   - row.index is now the column names from df.columns
   - row.name is the (individual) row-index value of that row in the parent dataframe

(I didn't look into the case with multi-index.).

[jreback]

@smcinerney jot opposed to a small update in the doc string but adding examples might be more useful

[MarcoGorelli]

@smcinerney feel free to submit a pull request

[smcinerney]

Guys can we please go with just this doc change? I think it's a big enough improvement. Don't need code examples IMO. (I'm aware of multiple other cases in pandas doc where code examples are badly needed, and this one ain't it.)

[jbrockmendel]

@smcinerney want to make a PR for this?