New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef #9693

Merged
merged 6 commits into from Sep 11, 2017

Conversation

Projects
None yet
5 participants
@sam-s
Contributor

sam-s commented Sep 5, 2017

Reference Issue

Fixes #9622

What does this implement/fix? Explain your changes.

avoid integer overflow by using floats

@jnothman

Could you please add a test?

Show outdated Hide outdated sklearn/metrics/classification.py

@jnothman jnothman changed the title from Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef to [MRG+1] avoid integer overflow by using floats for matthews_corrcoef Sep 6, 2017

@jnothman jnothman added this to the 0.19.1 milestone Sep 6, 2017

@jnothman

This comment has been minimized.

Show comment
Hide comment
@jnothman

jnothman Sep 6, 2017

Member

flake8 errors:


./sklearn/metrics/tests/test_classification.py:491:33: E231 missing whitespace after ','
        true_pos = conf_matrix[1,1]
                                ^
./sklearn/metrics/tests/test_classification.py:492:34: E231 missing whitespace after ','
        false_pos = conf_matrix[1,0]
                                 ^
./sklearn/metrics/tests/test_classification.py:493:34: E231 missing whitespace after ','
        false_neg = conf_matrix[0,1]
Member

jnothman commented Sep 6, 2017

flake8 errors:


./sklearn/metrics/tests/test_classification.py:491:33: E231 missing whitespace after ','
        true_pos = conf_matrix[1,1]
                                ^
./sklearn/metrics/tests/test_classification.py:492:34: E231 missing whitespace after ','
        false_pos = conf_matrix[1,0]
                                 ^
./sklearn/metrics/tests/test_classification.py:493:34: E231 missing whitespace after ','
        false_neg = conf_matrix[0,1]
@lesteve

Maybe the code can be made simpler and the test as well

Show outdated Hide outdated sklearn/metrics/classification.py
Show outdated Hide outdated sklearn/metrics/tests/test_classification.py
@sam-s

This comment has been minimized.

Show comment
Hide comment
@sam-s

sam-s Sep 7, 2017

Contributor

what does "MRG+1" mean?

Contributor

sam-s commented Sep 7, 2017

what does "MRG+1" mean?

@NelleV

This comment has been minimized.

Show comment
Hide comment
@NelleV

NelleV Sep 7, 2017

Member

@sam-s It means the pull requests has been approved by one core contributor.

Member

NelleV commented Sep 7, 2017

@sam-s It means the pull requests has been approved by one core contributor.

@jnothman

This comment has been minimized.

Show comment
Hide comment
@jnothman

jnothman Sep 7, 2017

Member

@lesteve I don't entirely get why your solution works. Isn't the largest number n_samples_square, which remains an integer in your code?

Member

jnothman commented Sep 7, 2017

@lesteve I don't entirely get why your solution works. Isn't the largest number n_samples_square, which remains an integer in your code?

@lesteve

This comment has been minimized.

Show comment
Hide comment
@lesteve

lesteve Sep 8, 2017

Member

@lesteve I don't entirely get why your solution works. Isn't the largest number n_samples_square, which remains an integer in your code?

Integer division behaves as we want for Python3 (i.e. true division) and we have from __future__ import division so that it behaves as we want as well for Python 2. Maybe more related to what you wanted to hear, the overflow was because we were multiplying two covariances (order n^2 * n^2 super roughly). For n^2 to overflow you need 3 billion samples.

Member

lesteve commented Sep 8, 2017

@lesteve I don't entirely get why your solution works. Isn't the largest number n_samples_square, which remains an integer in your code?

Integer division behaves as we want for Python3 (i.e. true division) and we have from __future__ import division so that it behaves as we want as well for Python 2. Maybe more related to what you wanted to hear, the overflow was because we were multiplying two covariances (order n^2 * n^2 super roughly). For n^2 to overflow you need 3 billion samples.

@sam-s

This comment has been minimized.

Show comment
Hide comment
@sam-s

sam-s Sep 8, 2017

Contributor

in the current patch, n_samples is a float64.

Contributor

sam-s commented Sep 8, 2017

in the current patch, n_samples is a float64.

@lesteve

This comment has been minimized.

Show comment
Hide comment
@lesteve

lesteve Sep 8, 2017

Member

given the lack of transparent bignums in numpy, your magic is dangerous.
I don't mind the other change ("manipulating quantities that are of order 1"), but casting to float is critical.

Although I agree it can be surprising at first I would not call it magic, it is just the behaviour of true division that operands get casted to floats before the division takes place.

I am not going to claim to be a floating point expert, but I would go with my version posted in #9693 (comment).

Member

lesteve commented Sep 8, 2017

given the lack of transparent bignums in numpy, your magic is dangerous.
I don't mind the other change ("manipulating quantities that are of order 1"), but casting to float is critical.

Although I agree it can be surprising at first I would not call it magic, it is just the behaviour of true division that operands get casted to floats before the division takes place.

I am not going to claim to be a floating point expert, but I would go with my version posted in #9693 (comment).

@sam-s

This comment has been minimized.

Show comment
Hide comment
@sam-s

sam-s Sep 8, 2017

Contributor
Contributor

sam-s commented Sep 8, 2017

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Sep 8, 2017

Member

At any rate, "manipulating quantities that are of order 1" does not
matter for floats (by the mere meaning of the term "floating point
number").

The exponent can also overflow, though I feel that is very unlikely to happen.

I'm more concerned about C.sum() to overflow (though also unlikely?), which is avoided by casting to float.

Member

amueller commented Sep 8, 2017

At any rate, "manipulating quantities that are of order 1" does not
matter for floats (by the mere meaning of the term "floating point
number").

The exponent can also overflow, though I feel that is very unlikely to happen.

I'm more concerned about C.sum() to overflow (though also unlikely?), which is avoided by casting to float.

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Sep 8, 2017

Member

Either solution seems fine for me.

Member

amueller commented Sep 8, 2017

Either solution seems fine for me.

@sam-s

This comment has been minimized.

Show comment
Hide comment
@sam-s

sam-s Sep 8, 2017

Contributor
Contributor

sam-s commented Sep 8, 2017

@jnothman

I'm happy to merge whichever solution

@sam-s

This comment has been minimized.

Show comment
Hide comment
@sam-s

sam-s Sep 10, 2017

Contributor
Contributor

sam-s commented Sep 10, 2017

@jnothman

This comment has been minimized.

Show comment
Hide comment
@jnothman

jnothman Sep 10, 2017

Member

We still need a second lgtm

Member

jnothman commented Sep 10, 2017

We still need a second lgtm

@amueller amueller merged commit 4c61e8b into scikit-learn:master Sep 11, 2017

6 checks passed

ci/circleci Your tests passed on CircleCI!
Details
codecov/patch 100% of diff hit (target 96.17%)
Details
codecov/project 96.17% (+<.01%) compared to fb64216
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
lgtm analysis: Python No alert changes
Details
@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Sep 11, 2017

Member

thanks

Member

amueller commented Sep 11, 2017

thanks

jnothman added a commit to jnothman/scikit-learn that referenced this pull request Sep 12, 2017

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef (#…
…9693)

* Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef

* matthews_corrcoef: cosmetic change requested by jnothman

* Add test_matthews_corrcoef_overflow for Bug#9622

* test_matthews_corrcoef_overflow: clean-up and make deterministic

* matthews_corrcoef: pass dtype=np.float64 to sum & trace instead of using astype

* test_matthews_corrcoef_overflow: add simple deterministic tests

amueller added a commit to amueller/scikit-learn that referenced this pull request Sep 12, 2017

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef (#…
…9693)

* Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef

* matthews_corrcoef: cosmetic change requested by jnothman

* Add test_matthews_corrcoef_overflow for Bug#9622

* test_matthews_corrcoef_overflow: clean-up and make deterministic

* matthews_corrcoef: pass dtype=np.float64 to sum & trace instead of using astype

* test_matthews_corrcoef_overflow: add simple deterministic tests

massich added a commit to massich/scikit-learn that referenced this pull request Sep 15, 2017

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef (#…
…9693)

* Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef

* matthews_corrcoef: cosmetic change requested by jnothman

* Add test_matthews_corrcoef_overflow for Bug#9622

* test_matthews_corrcoef_overflow: clean-up and make deterministic

* matthews_corrcoef: pass dtype=np.float64 to sum & trace instead of using astype

* test_matthews_corrcoef_overflow: add simple deterministic tests

amueller added a commit to amueller/scikit-learn that referenced this pull request Sep 19, 2017

Don't modify steps in Pipeline.__init__
remove outdated comment

fix also for FeatureUnion

[MRG+2] Limiting n_components by both n_features and n_samples instead of just n_features (Recreated PR) (#8742)

[MRG+1] Remove hard dependency on nose (#9670)

MAINT Stop vendoring sphinx-gallery (#9403)

CI upgrade travis to run on new numpy release (#9096)

CI Make it possible to run doctests in .rst files with pytest (#9697)

* doc/datasets/conftest.py to implement the equivalent of nose fixtures
* add conftest.py in root folder to ensure that sklearn local folder
  is used rather than the package in site-packages
* test doc with pytest in Travis
* move custom_data_home definition from nose fixture to .rst file

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef (#9693)

* Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef

* matthews_corrcoef: cosmetic change requested by jnothman

* Add test_matthews_corrcoef_overflow for Bug#9622

* test_matthews_corrcoef_overflow: clean-up and make deterministic

* matthews_corrcoef: pass dtype=np.float64 to sum & trace instead of using astype

* test_matthews_corrcoef_overflow: add simple deterministic tests

TST Platform independent hash collision tests in FeatureHasher (#9710)

TST More informative error message in test_preserve_trustworthiness_approximately (#9738)

add some rudimentary tests for meta-estimators

fix extra whitespace in error message

add missing if_delegate_has_method in pipeline

don't test tuple pipeline for now

only copy list if not list already? doesn't seem to help?

maskani-moh added a commit to maskani-moh/scikit-learn that referenced this pull request Nov 15, 2017

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef (#…
…9693)

* Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef

* matthews_corrcoef: cosmetic change requested by jnothman

* Add test_matthews_corrcoef_overflow for Bug#9622

* test_matthews_corrcoef_overflow: clean-up and make deterministic

* matthews_corrcoef: pass dtype=np.float64 to sum & trace instead of using astype

* test_matthews_corrcoef_overflow: add simple deterministic tests

jwjohnson314 pushed a commit to jwjohnson314/scikit-learn that referenced this pull request Dec 18, 2017

[MRG+1] avoid integer overflow by using floats for matthews_corrcoef (#…
…9693)

* Fix bug#9622: avoid integer overflow by using floats for matthews_corrcoef

* matthews_corrcoef: cosmetic change requested by jnothman

* Add test_matthews_corrcoef_overflow for Bug#9622

* test_matthews_corrcoef_overflow: clean-up and make deterministic

* matthews_corrcoef: pass dtype=np.float64 to sum & trace instead of using astype

* test_matthews_corrcoef_overflow: add simple deterministic tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment