BUG: Closes #5027 distance function always casts bool to double

[Garrett-R]

Closes #5027: Always casting to double probably incurred a performance hit for the boolean-based distance metrics such as the "matching distance".

I'm a new SciPy contributor, so let me know if I can do anything better.

Two questions:

1. Wasn't sure if I should add to the release notes since it seems like it was all done by rgommers in the [last release|https://github.com/scipy/scipy/commit/75c17a04530c2604a241209b1e1ed5c79c59cdff], but on the other hand the new developer docs [say you should add a release note|https://github.com/scipy/scipy/blame/master/HACKING.rst.txt#L243].

2. I didn't add a test since nothing about the functionality of this function (pdist) actually changed and I couldn't think how to test it. Is that fine?

[Garrett-R]

unused import.

[pv]

Sorry for the delay:
The following seem to also need _convert_to_double:

• _distance_wrap.pdist_city_block_wrap(X, dm) needs _convert_to_double(X)
• elif mstr in set(['cosine', 'cos']): (the calculations should cast to double to avoid overflow for integer inputs)
• elif mstr in set(['old_cosine', 'old_cos']):
• potentially also other places below

I would suggest the following for clarity: instead of

_distance_wrap.pdist_sqeuclidean_wrap(_convert_to_double(X), dm)

write

X = _convert_to_double(X)
_distance_wrap.pdist_sqeuclidean_wrap(X, dm)

and require that each elif branch must start with one of

X = _convert_to_double(X)
X = _convert_to_bool(X)

Then it's possible to check no casts are missing.

[pv]

The cdist routine also seems to have the same issue (not necessary to fix in this PR though).

[Garrett-R]

Removed this 7 year old commented code.

[Garrett-R]

@pv, right I agree with all those things you said. I also did the cdist function while I was at it.

This should be ready to (re-)review.

[larsmans]

This needs a rebase as #5665 got rid of the matching distance (because it's a duplicate of the Hamming distance).

[Garrett-R]

Thanks, @larsmans, I've rebased it.

[codecov-io]

@@            master   #5384   diff @@
======================================
  Files          235     235       
  Stmts        43211   43281    +70
  Branches      8163    8163       
  Methods          0       0       
======================================
+ Hit          33537   33609    +72
+ Partial       2602    2598     -4
- Missed        7072    7074     +2

Review entire Coverage Diff as of 3aeb659

Powered by Codecov. Updated on successful CI builds.

[larsmans]

I'm merging this as it's a big improvement. Thanks @Garrett-R!

[Garrett-R]

Thanks!