TST: Fix dtype mismatch on 32bit in IntervalTree get_indexer test

[jschendel]

xref #23442 (comment)

To address @jreback's comment in the xref:

.get_indexer should be returning platform int - but it’s not ? (maybe just this case)

It looks like IntervalTree.get_indexer and any other similar IntervalTree methods (get_loc, get_indexer_nonunique, etc.) will always return int64 as the data comes from a Int64Vector.to_array().

If returning platform int is expected, I'm not sure that making this change now is the best use of time: I'm hitting these methods as part of the new IntervalIndex behavior specs, so probably best enforce platform int as part of that implementation.

[pep8speaks]

Hello @jschendel! Thanks for submitting the PR.

• There are no PEP8 issues in the file pandas/tests/indexes/interval/test_interval_tree.py !

[jreback]

this just needs a small change in .get_indexer i think

this is inconsistent as II is the only index returning the wrong type

[jreback]

This is called in 1 case in get_indexer. I think this needs wrapping with ensure_platform_int (could be in get_indexer)

 def _find_non_overlapping_monotonic_bounds(self, key):
       if isinstance(key, IntervalMixin):
           start = self._searchsorted_monotonic(
               key.left, 'left', exclude_label=key.open_left)
           stop = self._searchsorted_monotonic(
               key.right, 'right', exclude_label=key.open_right)
       elif isinstance(key, slice):
           # slice
           start, stop = key.start, key.stop
           if (key.step or 1) != 1:
               raise NotImplementedError("cannot slice with a slice step")
           if start is None:
               start = 0
           else:
               start = self._searchsorted_monotonic(start, 'left')
           if stop is None:
               stop = len(self)
           else:
               stop = self._searchsorted_monotonic(stop, 'right')
       else:
           # scalar or index-like

           start = self._searchsorted_monotonic(key, 'left')
           stop = self._searchsorted_monotonic(key, 'right')
       return start, stop

[TomAugspurger]

Merged master, to see if the code-check failure in https://travis-ci.org/pandas-dev/pandas/jobs/450067817#L2756 goes away. It wasn't clear to me what the issue was.

[codecov]

Codecov Report

Merging #23468 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #23468      +/-   ##
==========================================
- Coverage   92.25%   92.24%   -0.01%     
==========================================
  Files         161      161              
  Lines       51181    51224      +43     
==========================================
+ Hits        47217    47254      +37     
- Misses       3964     3970       +6

Flag	Coverage Δ
#multiple	90.63% <ø> (-0.01%)	⬇️
#single	42.28% <ø> (-0.01%)	⬇️

Impacted Files	Coverage Δ
pandas/core/indexes/timedeltas.py	89.88% <0%> (-0.84%)	⬇️
pandas/core/arrays/base.py	97.35% <0%> (-0.67%)	⬇️
pandas/core/internals/blocks.py	93.67% <0%> (-0.36%)	⬇️
pandas/core/indexes/datetimes.py	96.13% <0%> (-0.29%)	⬇️
pandas/core/arrays/datetimelike.py	95.9% <0%> (-0.19%)	⬇️
pandas/core/arrays/categorical.py	95.09% <0%> (-0.13%)	⬇️
pandas/core/arrays/sparse.py	91.71% <0%> (-0.13%)	⬇️
pandas/io/json/json.py	93.11% <0%> (-0.02%)	⬇️
pandas/core/indexes/base.py	96.45% <0%> (-0.01%)	⬇️
pandas/core/reshape/reshape.py	99.54% <0%> (-0.01%)	⬇️
... and 26 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd98841...7e83460. Read the comment docs.

[jreback]

@jschendel are you able to fix this rather than this work-around?

[jschendel]

@jreback : will look into this tonight. sorry, been super bogged down at work.

[jschendel]

Modified IntervalTree methods to return 'intp' dtype arrays via an astype after calling Int64Vector.to_array.

[TomAugspurger]

Fixed the linting error (untested).

[jreback]

lgtm. let's merge on green.

[jreback]

thanks @jschendel and @TomAugspurger

[jreback]

still failing
https://travis-ci.org/MacPython/pandas-wheels/jobs/452707967
see end of log

[jschendel]

Yeah, of course changing the IntervalTree code to return platform int would cascade to other things. Pretty shortsighted of me. Should be an easy fix of 'int64' --> 'intp' in the expected though, so will get that in later today.

It looks like one of the failures is unrelated to my changes though:

_______________________ TestSparseGroupBy.test_aggfuncs ________________________
self = <pandas.tests.sparse.test_groupby.TestSparseGroupBy object at 0xde3f08ec>
    def test_aggfuncs(self):
        sparse_grouped = self.sparse.groupby('A')
        dense_grouped = self.dense.groupby('A')
    
        result = sparse_grouped.mean().to_sparse()
        expected = dense_grouped.mean().to_sparse()
    
>       tm.assert_frame_equal(result, expected)
/venv/lib/python3.6/site-packages/pandas/tests/sparse/test_groupby.py:50: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/venv/lib/python3.6/site-packages/pandas/util/testing.py:1185: in assert_extension_array_equal
    assert_numpy_array_equal(left_valid, right_valid)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
left = array([-0.17387645482451206, 0.3414148016424936], dtype=object)
right = array([-0.17387645482451206, 0.3414148016424937], dtype=object)
err_msg = None
    def _raise(left, right, err_msg):
        if err_msg is None:
            if left.shape != right.shape:
                raise_assert_detail(obj, '{obj} shapes are different'
                                    .format(obj=obj), left.shape, right.shape)
    
            diff = 0
            for l, r in zip(left, right):
                # count up differences
                if not array_equivalent(l, r, strict_nan=strict_nan):
                    diff += 1
    
            diff = diff * 100.0 / left.size
            msg = '{obj} values are different ({pct} %)'.format(
                obj=obj, pct=np.round(diff, 5))
>           raise_assert_detail(obj, msg, left, right)
E           AssertionError: numpy array are different
E           
E           numpy array values are different (50.0 %)
E           [left]:  [-0.17387645482451206, 0.3414148016424936]
E           [right]: [-0.17387645482451206, 0.3414148016424937]
/venv/lib/python3.6/site-packages/pandas/util/testing.py:1146: AssertionError

This error was not present in build #763 and then appeared in build #764; I had not made any changes between those two builds.