Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: Index.asi8 #37877

Merged
merged 9 commits into from
Nov 19, 2020
3 changes: 2 additions & 1 deletion doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -457,7 +457,8 @@ Deprecations
- :class:`Index` methods ``&``, ``|``, and ``^`` behaving as the set operations :meth:`Index.intersection`, :meth:`Index.union`, and :meth:`Index.symmetric_difference`, respectively, are deprecated and in the future will behave as pointwise boolean operations matching :class:`Series` behavior. Use the named set methods instead (:issue:`36758`)
- :meth:`Categorical.is_dtype_equal` and :meth:`CategoricalIndex.is_dtype_equal` are deprecated, will be removed in a future version (:issue:`37545`)
- :meth:`Series.slice_shift` and :meth:`DataFrame.slice_shift` are deprecated, use :meth:`Series.shift` or :meth:`DataFrame.shift` instead (:issue:`37601`)
- Partial slicing on unordered :class:`DatetimeIndexes` with keys, which are not in Index is deprecated and will be removed in a future version (:issue:`18531`)
- Partial slicing on unordered :class:`DatetimeIndex` with keys, which are not in Index is deprecated and will be removed in a future version (:issue:`18531`)
- Deprecated :meth:`Index.asi8` (:issue:`37877`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify this deprecation is only for the non-datetimelike Index subclasses for which the attribute currently returns None?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea, will update

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated + greenish


.. ---------------------------------------------------------------------------

Expand Down
16 changes: 11 additions & 5 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,11 @@ def asi8(self):
ndarray
An ndarray with int64 dtype.
"""
warnings.warn(
"Index.asi8 is deprecated and will be removed in a future version",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add that the replacement is .astype('i8')

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trouble is thats not really correct for the Index classes we're deprecating it for. For these classes .asi8 returns None

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh ok. are we deprecating for DTI and cousins? (I think we should do that as well to be consistent).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ill take a look; that might be pretty invasive

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok i think you should make this an internal method and deprecate the user facing one (even though we use it internally in the many parts of pandas)

FutureWarning,
stacklevel=2,
)
return None

@classmethod
Expand Down Expand Up @@ -4717,12 +4722,13 @@ def argsort(self, *args, **kwargs) -> np.ndarray:
>>> idx[order]
Index(['a', 'b', 'c', 'd'], dtype='object')
"""
result = self.asi8

if result is None:
result = np.array(self)
if needs_i8_conversion(self.dtype):
# TODO: these do not match the underlying EA argsort methods GH#37863
return self.asi8.argsort(*args, **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you calling .asi8 here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment just above #37863 a handful of tests fail if we pass through to the backing EA.argsort

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't these show the deprecation warning?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, because we're not deprecating it for the datetimelike classes


return result.argsort(*args, **kwargs)
# This works for either ndarray or EA, is overriden
# by RangeIndex, MultIIndex
return self._data.argsort(*args, **kwargs)

@final
def get_value(self, series: "Series", key):
Expand Down
6 changes: 4 additions & 2 deletions pandas/core/tools/numeric.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
is_number,
is_numeric_dtype,
is_scalar,
needs_i8_conversion,
)
from pandas.core.dtypes.generic import ABCIndexClass, ABCSeries

Expand Down Expand Up @@ -123,8 +124,9 @@ def to_numeric(arg, errors="raise", downcast=None):
values = arg.values
elif isinstance(arg, ABCIndexClass):
is_index = True
values = arg.asi8
if values is None:
if needs_i8_conversion(arg.dtype):
values = arg.asi8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't you deprecating this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only deprecating the one in pd.Index that returns None

else:
values = arg.values
elif isinstance(arg, (list, tuple)):
values = np.array(arg, dtype="O")
Expand Down
11 changes: 9 additions & 2 deletions pandas/core/window/rolling.py
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,13 @@ def _get_roll_func(self, func_name: str) -> Callable[..., Any]:
)
return window_func

@property
def _index_array(self):
# TODO: why do we get here with e.g. MultiIndex?
if needs_i8_conversion(self._on.dtype):
return self._on.asi8
return None

def _get_window_indexer(self) -> BaseIndexer:
"""
Return an indexer class that will compute the window start and end bounds
Expand All @@ -345,7 +352,7 @@ def _get_window_indexer(self) -> BaseIndexer:
return self.window
if self.is_freq_type:
return VariableWindowIndexer(
index_array=self._on.asi8, window_size=self.window
index_array=self._index_array, window_size=self.window
)
return FixedWindowIndexer(window_size=self.window)

Expand Down Expand Up @@ -2143,7 +2150,7 @@ def _get_window_indexer(self) -> GroupbyIndexer:
"""
rolling_indexer: Type[BaseIndexer]
indexer_kwargs: Optional[Dict[str, Any]] = None
index_array = self._on.asi8
index_array = self._index_array
window = self.window
if isinstance(self.window, BaseIndexer):
rolling_indexer = type(self.window)
Expand Down
23 changes: 22 additions & 1 deletion pandas/tests/indexes/test_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,16 @@
from pandas.core.dtypes.common import is_period_dtype, needs_i8_conversion

import pandas as pd
from pandas import CategoricalIndex, MultiIndex, RangeIndex
from pandas import (
CategoricalIndex,
DatetimeIndex,
Int64Index,
MultiIndex,
PeriodIndex,
RangeIndex,
TimedeltaIndex,
UInt64Index,
)
import pandas._testing as tm


Expand Down Expand Up @@ -348,6 +357,18 @@ def test_ravel_deprecation(self, index):
with tm.assert_produces_warning(FutureWarning):
index.ravel()

def test_asi8_deprecation(self, index):
# GH#37877
if isinstance(
index, (Int64Index, UInt64Index, DatetimeIndex, TimedeltaIndex, PeriodIndex)
):
warn = None
else:
warn = FutureWarning

with tm.assert_produces_warning(warn):
index.asi8


@pytest.mark.parametrize("na_position", [None, "middle"])
def test_sort_values_invalid_na_position(index_with_missing, na_position):
Expand Down