Skip to content

Fix error in euclidean_distances when X is float64 and X_norm_squared is float32 - #27624

Merged
betatim merged 5 commits into
scikit-learn:mainfrom
jeromedockes:fix_27621
Oct 26, 2023
Merged

betatim merged 5 commits into
scikit-learn:mainfrom
jeromedockes:fix_27621

Conversation

@jeromedockes

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Fixes #27621

What does this implement/fix? Explain your changes.

euclidean_distances discards precomputed squared norms if they are in float32 to avoid numerical precision issues.
But in the current implementation there is a code path where the used squared distances end up being None, when the X and Y are float64 but the squared distances are float32.

This PR implements the following logic

  • if X_norm_squared is provided in float64: use it
  • otherwise if X is float64: use it to compute the squared norm
  • otherwise rely on _euclidean_distances_upcast (as is done ATM when X is float32)

and the same for Y

my impression is that this was the original intention but maybe @jeremiedbb can confirm

Any other comments?

@github-actions

github-actions Bot commented Oct 19, 2023 •

Copy link
Copy Markdown

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: ee65beb. Link to the linter CI: here

@jjerphan jjerphan added the good first PR to review Simple atomic PR to review label Oct 20, 2023

@jjerphan jjerphan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you, @jeromedockes.

Can you add a |Fix| changelog entry?

Comment thread sklearn/metrics/tests/test_pairwise.py Outdated
jeromedockes and others added 2 commits October 23, 2023 10:11
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

@jeremiedbb jeremiedbb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the fix

@dineshchitlangia dineshchitlangia left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, non binding.

@betatim
betatim merged commit 93fa00c into scikit-learn:main Oct 26, 2023
@betatim

betatim commented Oct 26, 2023

Copy link
Copy Markdown
Member

Thanks for the fix!

glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Oct 31, 2023
…red` is float32 (scikit-learn#27624)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
REDVM pushed a commit to REDVM/scikit-learn that referenced this pull request Nov 16, 2023
…red` is float32 (scikit-learn#27624)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

euclidean_distances with float64 x,y and float32 xx and yy

5 participants