Use Index._column in favor of Index._values

[mroeschke]

Description

The base definition is

    @property  # type: ignore
    @_cudf_nvtx_annotate
    def _values(self):
        return self._column

So I think it's just more straightforward to use self._columns

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[vyasr]

This change is somewhat conceptually problematic. A RangeIndex isn't backed by a column. My hope was to be able to rewrite algorithms in cudf to never rely on this kind of method at all, so as to be agnostic to the type of index and not accidentally materialize GPU data (see #9593). Should we prioritize those changes rather than making a cosmetic improvement like this one that is likely to only increase the frequency with which we accidentally allocate device memory?

[mroeschke]

An effort to remove BaseIndex._values entirely as described in #9593 seems like a more worthy undertaking.

Curious how you envisioned intentionally materializing a RangeIndex to a column? as_column?

[vyasr]

I think a good starting point would be removing the ability to materialize a RangeIndex (i.e. removing those code paths from as_column and methods like RangeIndex._values altogether) and then seeing what breaks. In an ideal world we would implement versions of our code that avoid materialization altogether.

[mroeschke]

OK some initial operations that would break:

1. .reset_index where the index is a RangeIndex i.e. the RangeIndex needs to materialized to a new column
2. RangeIndex.equals e.g. an Index[int64] can equals a RangeIndex. (I suppose materialization can be avoided by checking if an Index can be "compressed" to a range)
3. RangeIndex.astype to a non int64 type

[vyasr]

1. My gut reaction is that this seems wrong. If the index is a RangeIndex, then the output of reset_index should also be a RangeIndex. pandas bears this out

>>> df = pd.DataFrame({'a': [1, 2, 3]})
>>> df.index
RangeIndex(start=0, stop=3, step=1)
>>> df.reset_index().index
RangeIndex(start=0, stop=3, step=1)

where does it break for us?
2. I think figuring out whether an integer index can be "compressed" to a range is going to be more work to implement than it's worth. To do that without allocating any intermediate memory for the diffs would require a slightly nontrivial algorithm to do something like compute the steps between every pair of elements, then do some synchronizations of a local variable a la a reduction. A simpler approach that would still require some GPU code but would be conceptually very straightforward would be to implement a libcudf function that allows you to check whether a column is equal to a range by specifying the range as a smart iterator, e.g. a counting iterator from start to stop with the step. That would be almost trivial to implement in C++. I don't think it's worth doing this right now since there would be some additional work to make that available in Python and then branch accordingly in Column.equals instead of calling a binop, but maybe we open an issue with the suggestion and leave it as a good first issue for libcudf (unless you're interested in tackling it yourself!).
3. We obviously can't avoid materializing in this case, but can we change how we handle this? For example, here's a one-liner that gets us what we need and would allow us to never need to use RangeIndex._values:

>>> df = pd.DataFrame({'a': [1, 2, 3]})
>>> df.index
RangeIndex(start=0, stop=3, step=1)
>>> cp.asarray(df.index, dtype=np.int32)
array([0, 1, 2], dtype=int32)

[vyasr]

@mroeschke should we close this PR? Is it useful for tracking purposes? IIRC based on the main discussion above we're going in a different direction altogether to try and reduce materializations.

[mroeschke]

It's useful for tracking but it doesn't need to remain open so I'll close this