Skip to content

Conversation

arthurlw
Copy link
Member

@arthurlw arthurlw commented Mar 21, 2025

@arthurlw arthurlw changed the title Pass boolean to iloc. getitem Support boolean masks for iloc.__getitem__ Mar 21, 2025
@arthurlw arthurlw changed the title Support boolean masks for iloc.__getitem__ BUG: Support boolean masks for iloc.__getitem__ Mar 21, 2025
@arthurlw arthurlw changed the title BUG: Support boolean masks for iloc.__getitem__ ENH: Support boolean masks for iloc.__getitem__ Mar 21, 2025
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Am I correct that none of the tests being modified here actually call .iloc successfully?

}

# UserWarnings from reindex of a boolean mask
for idx in [None, "index", "locs"]:
mask = (df.nums > 2).values
if idx:
mask_index = getattr(df, idx)[::-1]
mask_index = getattr(df, idx if idx == "index" else "locs")[::-1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since idx is not None here, I think this can be reverted.

Comment on lines 779 to 782
except (
ValueError,
IndexingError,
) as err:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can you collapse this back to one line

@rhshadrach rhshadrach added Enhancement Indexing Related to indexing on series/frames, not to indexes themselves labels Mar 29, 2025
@arthurlw
Copy link
Member Author

arthurlw commented Mar 29, 2025

Hi! The test on lines 742-743 calls .iloc with a boolean mask successfully:

result = df.iloc[np.array([True] * len(mask), dtype=bool)]
tm.assert_frame_equal(result, df)

I noticed I should add another test with false values so I have done so.

result2 = df.iloc[np.array([True, False, True, False, True], dtype=bool)]
tm.assert_frame_equal(result2, DataFrame({"a": [0, 2, 4]}, index=["A", "C", "E"]))

@arthurlw
Copy link
Member Author

arthurlw commented Apr 5, 2025

While reviewing this, I realized my tests address .iloc when passing boolean masks rather than Series[bool] as #60994 originally intended. I will be updating my tests accordingly.

That being said the new tests improve coverage for boolean mask behavior for .iloc, which I think is helpful.

@arthurlw arthurlw changed the title ENH: Support boolean masks for iloc.__getitem__ ENH: Support Series[bool] as indexer for iloc.__getitem__ Apr 7, 2025
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm; @mroeschke - could I get your eye here as well.

Comment on lines 745 to 763
result = df.iloc[np.array([True, False, True, False, True], dtype=bool)]
tm.assert_frame_equal(
result, DataFrame({"a": [0, 2, 4]}, index=["A", "C", "E"])
)

# series (index does not match)
msg = "Unalignable boolean Series provided as indexer"
with pytest.raises(IndexingError, match=msg):
df.iloc[Series([True] * len(mask), dtype=bool)]

df = DataFrame(list(range(5)), columns=["a"])

result = df.iloc[Series([True] * len(mask), dtype=bool)]
tm.assert_frame_equal(result, df)

result = df.iloc[Series([True, False, True, False, True], dtype=bool)]
tm.assert_frame_equal(result, DataFrame({"a": [0, 2, 4]}, index=[0, 2, 4]))

df = DataFrame(list(range(5)), index=list("ABCDE"), columns=["a"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make each of these assertions a separate test function?

@mroeschke mroeschke added this to the 3.0 milestone Apr 9, 2025
@mroeschke mroeschke merged commit 214e474 into pandas-dev:main Apr 9, 2025
42 checks passed
@mroeschke
Copy link
Member

Thanks @arthurlw

snitish pushed a commit to snitish/pandas that referenced this pull request Jun 9, 2025
…-dev#61162)

* updated indexing.py to allow iloc.__getitem__

* Updated test_iloc_mask test

* bugfix test_iloc_mask test

* bugfix test_iloc_mask

* whatsnew

* added test to test_iloc_mask

* formatting

* precommit

* added tests for series bool mask

* precommit

* reformatted tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: iloc with Series as indexer fails for __getitem__ but works with __setitem__
3 participants