Skip to content

Commit

Permalink
Correct check when slicing non-monotonic datetime indexes
Browse files Browse the repository at this point in the history
The intention of pandas-dev#37819 was to deprecate (removed in pandas-dev#49607) the
special case behaviour of non-monotonic datetime indexes, so that if
either slice bound is not in the index, a KeyError is raised.

However, the check only fired correctly for the case where the lower
bound was not in the index and either the upper bound was None or it
was _also_ not in the index.

Correct the logic here and adapt the one test that exercises this
behaviour.

Closes pandas-dev#53983.
  • Loading branch information
wence- committed Jul 5, 2023
1 parent 172b7c1 commit e211d74
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 5 deletions.
8 changes: 4 additions & 4 deletions pandas/core/indexes/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -666,18 +666,18 @@ def check_str_or_none(point) -> bool:
return Index.slice_indexer(self, start, end, step)

mask = np.array(True)
raise_mask = np.array(True)
in_index = True
if start is not None:
start_casted = self._maybe_cast_slice_bound(start, "left")
mask = start_casted <= self
raise_mask = start_casted == self
in_index &= (start_casted == self).any()

if end is not None:
end_casted = self._maybe_cast_slice_bound(end, "right")
mask = (self <= end_casted) & mask
raise_mask = (end_casted == self) | raise_mask
in_index &= (end_casted == self).any()

if not raise_mask.any():
if not in_index:
raise KeyError(
"Value based partial slicing on non-monotonic DatetimeIndexes "
"with non-existing keys is not allowed.",
Expand Down
11 changes: 10 additions & 1 deletion pandas/tests/indexing/test_partial.py
Original file line number Diff line number Diff line change
Expand Up @@ -664,5 +664,14 @@ def test_slice_irregular_datetime_index_with_nan(self):
index = pd.to_datetime(["2012-01-01", "2012-01-02", "2012-01-03", None])
df = DataFrame(range(len(index)), index=index)
expected = DataFrame(range(len(index[:3])), index=index[:3])
result = df["2012-01-01":"2012-01-04"]
with pytest.raises(KeyError, match="non-existing keys is not allowed"):
# Upper bound is not in index (which is unordered)
# GH53983
# GH37819
df["2012-01-01":"2012-01-04"]
# Need this precision for right bound since the right slice
# bound is "rounded" up to the largest timepoint smaller than
# the next "resolution"-step of the provided point.
# e.g. 2012-01-03 is rounded up to 2012-01-04 - 1ns
result = df["2012-01-01":"2012-01-03 00:00:00.000000000"]
tm.assert_frame_equal(result, expected)

0 comments on commit e211d74

Please sign in to comment.