DatetimeIndex._maybe_cast_slice_bound with duplicates #16515

Closed
TomAugspurger opened this Issue May 26, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@TomAugspurger
Contributor

TomAugspurger commented May 26, 2017

DatetimeIndex._maybe_cast_slice_bound can return the wrong value when you have an index that is monotonically decreasing, but not strictly monotonically decreasing.

In [1]: import pandas as pd

In [2]: pd.DatetimeIndex(['2017', '2017'])
Out[2]: DatetimeIndex(['2017-01-01', '2017-01-01'], dtype='datetime64[ns]', freq=None)

In [3]: pd.DatetimeIndex(['2017', '2017'])._maybe_cast_slice_bound('2017-01-01', 'left', 'loc')
Out[3]: Timestamp('2017-01-01 23:59:59.999999999')

The issue is here That check really needs to check for strict monotonic decreasing, not just monotonic decreasing.

Have we hit any other needs for strict monotonic algos? This is such an extreme edge case, not sure it's worthwhile on its own.

Came up in dask/dask#2389

@TomAugspurger

This comment has been minimized.

Show comment
Hide comment
@TomAugspurger

TomAugspurger May 29, 2017

Contributor

Actually, we don't need any additional cython code. We can just do

@property
def is_strictly_monotonically_increasing(self):
    return self.is_unique and self.is_monotonic_increasing

Do we want to add that as a method? I'm slightly in favor of adding it.

Contributor

TomAugspurger commented May 29, 2017

Actually, we don't need any additional cython code. We can just do

@property
def is_strictly_monotonically_increasing(self):
    return self.is_unique and self.is_monotonic_increasing

Do we want to add that as a method? I'm slightly in favor of adding it.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback May 31, 2017

Contributor

@TomAugspurger yeah that sound fine (add tests as well), do this in base and it should just work for all Index sub-classes (maybe add is_strictly_monotonicaly_decreasing as well)

Contributor

jreback commented May 31, 2017

@TomAugspurger yeah that sound fine (add tests as well), do this in base and it should just work for all Index sub-classes (maybe add is_strictly_monotonicaly_decreasing as well)

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017

BUG: Fixed partial slicing on monotonic index with dupes
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017

BUG: Fixed partial slicing on monotonic index with dupes
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017

BUG: Fixed partial slicing on monotonic index with dupes
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017

BUG: Fixed partial slicing on monotonic index with dupes
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389

@jreback jreback closed this in #16555 Jun 1, 2017

@jreback jreback modified the milestones: 0.20.2, Next Major Release Jun 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment