Fix overflow error in cartesian_product

[david-hoffman]

When the numbers in X are large it can cause an overflow error on windows machine where the native int is 32 bit. Switching to np.intp alleviates this problem.

Other fixes would include switching to np.uint32 or np.uint64.

#15234

[codecov-io]

Codecov Report

Merging #15265 into 0.19.x will not impact coverage.

@@           Coverage Diff           @@
##           0.19.x   #15265   +/-   ##
=======================================
  Coverage   85.27%   85.27%           
=======================================
  Files         144      144           
  Lines       50946    50946           
=======================================
  Hits        43444    43444           
  Misses       7502     7502

Impacted Files	Coverage Δ
pandas/tools/util.py	96.77% <100%> (ø)	✅

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ead784f...c9c8d5e. Read the comment docs.

[jreback]

need a test for this (IOW something simple that fails w/o the fix and passes with it).

best way is to step thru your code until you hit this and see what the inputs are (to cartesian product) and can use that as a test.

need a whats new (bug fix) as well.

[jreback]

hmm, this is still quite large.

[david-hoffman]

I couldn't think of another way to cause the error without adding checking code to the cartesian_product function itself, e.g.

if np.any(cumprodX < 0):
    raise RuntimeError(...)

[jreback]

so, let's take the test out (just verify what you were doing works on windows).

pls add a whatsnew note and can merge

note that this is still buggy, because we do the cartesian product in the first place (this is #14942), so I think this will still break for you (just in a different place now).

[david-hoffman]

Yeah, I switched to an altered version of numpy's histogramdd for this problem.

[jreback]

ping on green.

[jreback]

thanks @david-hoffman

FYI, you were actually branched off of the 0.19.x branch, rather than master, but I picked the commit.