[MRG] Weights parameter of datasets.make_classification changed to array-like from list only - Issue 14760 #14764

CatChenal · 2019-08-24T17:02:34Z

Fixes #14760

What does this implement/fix?

The weights parameter can be a list or array, not just a list, e.g.
" weights is array-like or None " in docstring.

… (#wimlds)

…g (#wimlds)

…addition of # doctest: +NORMALIZE_WHITESPACE +DONT_ACCEPT_BLANKLINE +ELLIPSIS for print statements.

…addition of # doctest: +NORMALIZE_WHITESPACE +DONT_ACCEPT_BLANKLINE for print statements.

@reshamas

…DONT_ACCEPT_BLANKLINE (@jnothmam, @reshamas)

…xample.

amueller · 2019-08-24T17:38:43Z

sklearn/datasets/samples_generator.py

@@ -162,14 +162,14 @@ def make_classification(n_samples=100, n_features=20, n_informative=2,
    if n_informative < np.log2(n_classes * n_clusters_per_class):
        raise ValueError("n_classes * n_clusters_per_class must"
                         " be smaller or equal 2 ** n_informative")
-    if weights and len(weights) not in [n_classes, n_classes - 1]:
+    if all(weights) and len(weights) not in [n_classes, n_classes - 1]:


maybe weights is not None would be more clear?

amueller · 2019-08-24T17:38:51Z

sklearn/feature_extraction/image.py

@@ -337,22 +337,37 @@ def extract_patches_2d(image, patch_size, max_patches=None, random_state=None):

    Examples
    --------
+<<<<<<< HEAD


merge issues here

amueller · 2019-08-24T20:06:40Z

sklearn/datasets/samples_generator.py

@@ -162,17 +162,18 @@ def make_classification(n_samples=100, n_features=20, n_informative=2,
    if n_informative < np.log2(n_classes * n_clusters_per_class):
        raise ValueError("n_classes * n_clusters_per_class must"
                         " be smaller or equal 2 ** n_informative")
-    if weights and len(weights) not in [n_classes, n_classes - 1]:
-        raise ValueError("Weights specified but incompatible with number "
+    if not weights is None:


the idiomatic python way is weights is not None.
I would use a single if and and. You can put the whole condition in parenthesis and make it multi-line

amueller · 2019-08-24T20:06:58Z

sklearn/feature_extraction/image.py

@@ -337,22 +337,22 @@ def extract_patches_2d(image, patch_size, max_patches=None, random_state=None):

    Examples
    --------
-    >>> from sklearn.datasets import load_sample_image
+    >>> from sklearn.datasets import load_sample_images


this change seems unrelated

ping @CatChenal can you please revert these changes

amueller · 2019-08-24T20:07:17Z

Please add a non-regression test that would fail at master but pass in this PR.

…nerator.py

NicolasHug

Thanks for the PR @CatChenal ,

made a few comments

NicolasHug · 2019-08-26T18:29:20Z

sklearn/datasets/samples_generator.py

@@ -91,7 +91,7 @@ def make_classification(n_samples=100, n_features=20, n_informative=2,
    n_clusters_per_class : int, optional (default=2)
        The number of clusters per class.

-    weights : list of floats or None (default=None)
+    weights : sequence of floats or None (default=None)


We call this an array-like.

Suggested change

weights : sequence of floats or None (default=None)

weights : array-like of shape (n_classes,) or (n_classes - 1,), default=None

NicolasHug · 2019-08-26T18:29:20Z

sklearn/datasets/samples_generator.py

@@ -162,17 +162,18 @@ def make_classification(n_samples=100, n_features=20, n_informative=2,
    if n_informative < np.log2(n_classes * n_clusters_per_class):
        raise ValueError("n_classes * n_clusters_per_class must"
                         " be smaller or equal 2 ** n_informative")
-    if weights and len(weights) not in [n_classes, n_classes - 1]:
+    w_ok = (weights is not None) and all(weights)


shouldn't an error be raised if all(weights) is false?

NicolasHug · 2019-08-26T18:29:20Z

sklearn/datasets/samples_generator.py

-
-    if weights is None:
+    if weights is not None:
+        if all(weights) and len(weights) == (n_classes - 1):


Suggested change

if all(weights) and len(weights) == (n_classes - 1):

if len(weights) == (n_classes - 1):

NicolasHug · 2019-08-26T18:29:20Z

sklearn/datasets/samples_generator.py

+    if weights is not None:
+        if all(weights) and len(weights) == (n_classes - 1):
+            weights = weights + [1.0 - sum(weights)]
+    else:
        weights = [1.0 / n_classes] * n_classes
        weights[-1] = 1.0 - sum(weights[:-1])


That's not you, but that line is useless ;)

Which line?
Line 175 resizes the (n-classes - 1) array with the missing weight, so it makes sense.
Line 178 recalculate to last position of weights according to values set on line 177;
Line 178 is the useless, no?

weights = [1.0 / n_classes] * n_classes weights[-1] = 1.0 - sum(weights[:-1]) # <-- this one

NicolasHug · 2019-08-26T18:29:20Z

sklearn/datasets/tests/test_samples_generator.py

+    # w as array: should pass in PR_14764, fail in master
+    w = np.array([0.25, 0.75])
+    X, y = make_classification(weights=w)
+    assert X.shape == (100, 20), "X shape mismatch"


We like to parametrize these kind of tests. You can look for some inspiration in e.g. this test

Let us know if you need any help

None of the tests in test_samples_generator.py are parametrized. Do you want me to parametrize all of them or just the tests for make_classification()?

Oops, found one: test_make_blobs_n_samples_centers_none()

I was just suggesting to parametrize the test you wrote

Good...
I just found out one wrong way to do it:

@pytest.mark.parametrize( 'params, err_msg', [({'weights': 0}, "object of type 'int' has no len()"), ({'weights': -1}, "object of type 'int' has no len()"), ({'weights': []}, "Weights specified but incompatible with number of classes."), ({'weights': [.25,.75,.1]}, "Weights specified but incompatible with number of classes."), ({'weights': np.array([])},"Weights specified but incompatible with number of classes."), ({'weights': np.array([.25,.75,.1])},"Weights specified but incompatible with number of classes.")] ) def test_make_classification_weights_type(params, err_msg): make = partial(make_classification, n_samples=100, n_features=20, n_informative=2, n_redundant=2, n_repeated=0, n_classes=2, n_clusters_per_class=2, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=0) for i in range(len(params)): with pytest.raises(ValueError, match=err_msg[i]): make(weights=params[i]['weights'])

The first problem is that the mark.parametrize statement is incorrect: the weights in the partial functions are not split and I have not found out how to fix it yet.
The other problem is likely the iteration of the context manager (should not be needed...).
Thanks for pointing me in the right direction.

You're almost there. Here is a basic example:

@pytest.mark.parametrize( 'weights, err_msg', [ ([1, 2, 3], "incompatible with number of classes"), # add other test cases here ] ) def test_make_classification_weights_type(weights, err_msg): with pytest.raises(ValueError, match=err_msg): make_classification(weights=weights)

NicolasHug

Thanks @CatChenal , I made a few more comments but mostly looks good. Could you also please add a very simple test that makes sure passing e.g. [1, 2, 3] gives the same result as passing np.array([1, 2. 3]). Thanks!

NicolasHug · 2019-09-03T16:48:21Z

sklearn/feature_extraction/image.py

@@ -337,22 +337,22 @@ def extract_patches_2d(image, patch_size, max_patches=None, random_state=None):

    Examples
    --------
-    >>> from sklearn.datasets import load_sample_image
+    >>> from sklearn.datasets import load_sample_images


ping @CatChenal can you please revert these changes

NicolasHug · 2019-09-03T16:48:21Z

sklearn/datasets/samples_generator.py

@@ -380,7 +390,7 @@ def sample_example():
    X_indices = array.array('i')
    X_indptr = array.array('i', [0])
    Y = []
-    for i in range(n_samples):
+    for _ in range(n_samples):


Please avoid unrelated changes

NicolasHug · 2019-09-03T16:48:21Z

sklearn/datasets/samples_generator.py

@@ -1237,7 +1247,7 @@ def make_spd_matrix(n_dim, random_state=None):
    generator = check_random_state(random_state)

    A = generator.rand(n_dim, n_dim)
-    U, s, V = linalg.svd(np.dot(A.T, A))
+    U, _, V = linalg.svd(np.dot(A.T, A))


NicolasHug · 2019-09-03T16:48:21Z

sklearn/datasets/tests/test_samples_generator.py

@@ -11,6 +11,8 @@
 from sklearn.utils.testing import assert_array_almost_equal
 from sklearn.utils.testing import assert_raise_message

+from sklearn.utils.validation import assert_all_finite


NicolasHug · 2019-09-03T16:48:21Z

sklearn/datasets/samples_generator.py

+                                    n_informative, 2**n_informative))
+
+    if weights is not None:
+        if isinstance(weights, int):


I don't think we need a specific check for int (else it means we would need specific checks for pretty much every type). I guess a safe way is to convert the weights to a numpy array. You can then just check the length as you do below, and use np.sum everywhere.

NicolasHug · 2019-09-03T16:48:21Z

sklearn/datasets/samples_generator.py

+        if len(weights) not in [n_classes, n_classes - 1]:
+            raise ValueError("Weights specified but incompatible with number "
+                             "of classes.")
+        if len(weights) == (n_classes - 1):


Suggested change

if len(weights) == (n_classes - 1):

if len(weights) == n_classes - 1:

…return_sequences`.

NicolasHug

Minor comment about test coverage, but LGTM anyway. Thanks @CatChenal !

NicolasHug · 2019-09-04T12:18:03Z

sklearn/datasets/samples_generator.py

+            if isinstance(weights, list):
+                weights = weights + [1.0 - sum(weights)]
+            else:
+                weights = np.resize(weights, n_classes)


That part isn't covered by the tests. I think you can cover it easily by setting n_classes=3 in test_make_classification_weights_array_or_list_ok.

@NicolasHug

…ts_array_or_list_ok` as per @NicolasHug.

thomasjpfan

Thank you @CatChenal for working on this!

thomasjpfan · 2019-09-04T16:38:56Z

sklearn/datasets/tests/test_samples_generator.py

+                                 random_state=0)
+    X2, y2 = make_classification(weights=np.array([.1, .9]),
+                                 random_state=0)
+    assert (X1.all() == X2.all()) and (y1.all() == y2.all())


X1.all() returns True if X1 is all non-zero. Is this assertion to do the following:

assert_almost_equal(X1, X2) assert_almost_equal(y1, y2)

thomasjpfan · 2019-09-04T16:38:56Z

sklearn/datasets/tests/test_samples_generator.py

+        make_classification(weights=weights)
+
+
+def test_make_classification_weights_array_or_list_ok():


This can be parametrized:

@pytest.mark.parametrize("kwargs", [{}, {"n_classes": 3, "n_informative": 3}]) def test_make_classification_weights_array_or_list_ok(kwargs): X1, y1 = make_classification(weights=[.1, .9], random_state=0, **kwargs) X2, y2 = make_classification(weights=np.array([.1, .9]), random_state=0, **kwargs) assert_almost_equal(X1, X2) assert_almost_equal(y1, y2)

…thomasjpfan

…thomasjpfan

thomasjpfan · 2019-09-05T00:04:17Z

sklearn/datasets/samples_generator.py

@@ -91,7 +91,8 @@ def make_classification(n_samples=100, n_features=20, n_informative=2,
    n_clusters_per_class : int, optional (default=2)
        The number of clusters per class.

-    weights : list of floats or None (default=None)
+    weights : array-like of shape (n_classes,) or (n_classes - 1,),


Currently this is not rendered nicely.

To render nicely:

Suggested change

weights : array-like of shape (n_classes,) or (n_classes - 1,),

weights : array-like of shape (n_classes,) or (n_classes - 1,),\

Thanks, @thomasjpfan.
Would you please document how you reached that end-point to verify the rendering? My doc tree does not have a /modules/generated/ path.

When you build the html documentation using these instructions, there will be a new folder: doc/_build which contains doc/_build/html/stable/index.html which is the landing page of the scikit-learn. From there you can navigate to the make_classification docs by going to the API page.

thomasjpfan · 2019-09-05T04:09:09Z

Please add Enhancement entry to the change log at doc/whats_new/v0.22.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:.

… is not a linkable target.

thomasjpfan · 2019-09-05T18:57:54Z

doc/whats_new/v0.22.rst

@@ -85,6 +85,11 @@ Changelog
  :func:`datasets.fetch_20newsgroups` and :func:`datasets.fetch_olivetti_faces`
  . :pr:`14259` by :user:`Sourav Singh <souravsingh>`.

+- |Enhancement| `make_classification` in :func:`datasets.samples_generator` now 


Suggested change

- |Enhancement| `make_classification` in :func:`datasets.samples_generator` now

- |Enhancement| :func:`datasets.make_classification` now

thomasjpfan · 2019-09-05T18:57:54Z

doc/whats_new/v0.22.rst

+- |Enhancement| `make_classification` in :func:`datasets.samples_generator` now 
+  accepts array-like `weights` parameter, i.e. list or numpy.array, instead of 
+  list only.
+  :pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to `WiMLDS <WiMLDS>`.


Suggested change

:pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to `WiMLDS <WiMLDS>`.

:pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to *WiMLDS*.

Certainly not. That would downgrade @wimlds's contributions to Opensource & Scikit-learn in particular.

This would link you and @wimlds

- |Enhancement| :func:`datasets.make_classification` now accepts array-like `weights` parameter, i.e. list or numpy.array, instead of list only. :pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to :user:`WiMLDS <WiMLDS>`.

Certainly not. That would downgrade @wimlds's contributions to Opensource & Scikit-learn in particular.

Sorry, I misunderstood the intend of the string. I can see now you were trying to link to the organization on github. The above snippet should correctly link to their organization.

I'm not sure about this, we usually acknowledge individuals not organizations in release notes. For funding organizations are typically mentioned in https://scikit-learn.org/stable/about.html#funding). I think we could maybe add a section for WiMLDS and similar partner non-profit organizations there? The problem with acknowledgements of organizations in release notes is that most contributions have some sort of organization behind it ( Numfocus, conference where the sprint happened, company who allowed its employee to contribute during work time, etc), and then deciding to acknowledge some but not others is tricky.

Thanks for the information. The mention is then out of place in the release notes. I will remove it & add #WiMLDS in the final commit.

reshamas · 2019-09-09T15:24:49Z

@NicolasHug
Does this title need to include "MRG" ?

cc: @kellycarmody

jnothman · 2019-09-09T20:54:36Z

Is WiMLDS being listed here as a sponsor, or rather a way that the contributor was able to learn to contribute? I like the WiMLDS mention in the change log.

reshamas · 2019-09-09T21:02:53Z

@jnothman

Is WiMLDS being listed here as a sponsor, or rather a way that the contributor was able to learn to contribute? I like the WiMLDS mention in the change log.

WiMLDS contributed in the following ways:

organized the event: provided a way for people to contribute
sponsored the event: funding
sprint contributors
donated to NumFOCUS, for scikit-learn

Any way that is acknowledged would be cool.

NicolasHug · 2019-09-18T14:46:12Z

@thomasjpfan @rth comments were addressed it seems, let's merge?

NicolasHug · 2019-09-18T19:03:26Z

Thanks @CatChenal !!

CatChenal · 2019-09-18T19:59:44Z

Thank you @NicolasHug and @thomasjpfan!

CatChenal added 21 commits Dec 18, 2018

Finalizes fix for scikit-learn#12202 from abandonned PR by @parul-l

5cb4bda

Completes 12202 fix abandoned by @parul-l

a39baed

Completes 12202 fix abandoned by @parul-l

a5b73bb

Extends 12202 fix over feature_extraction/image.py

e9f9594

Closes scikit-learn#12202; white space removal

35da14a

Closes scikit-learn#12202; white space removal2

0730c7a

Closes scikit-learn#12202; 3.5 compliance; added >>> in docstring code.

f81beda

Closes scikit-learn#12202; indentation discrep.

294ce61

Closes scikit-learn#12202; indentation discrep.2

4f12330

Example output formating; @jnotham

2a8d782

Example output formating; forgot flake8

d14b3b6

Closes scikit-learn#12202; Removed excessive indentation in docstring…

3f8b0b1

… (#wimlds)

conflict resolution?

5fc5496

Closes scikit-learn#12202; Fixed inconsistent indentation in docstrin…

236dafb

…g (#wimlds)

Closes scikit-learn#12202 (#wimlds); intentation, v3.5 compliance

0bbbea4

Closes scikit-learn#12202 (#wimlds); Output format issue solved with …

f173bf6

…addition of # doctest: +NORMALIZE_WHITESPACE +DONT_ACCEPT_BLANKLINE +ELLIPSIS for print statements.

Closes scikit-learn#12202 (#wimlds); Output format issue solved with …

aeae3b0

…addition of # doctest: +NORMALIZE_WHITESPACE +DONT_ACCEPT_BLANKLINE for print statements.

Closes scikit-learn#12202 (#wimlds); Testing doctest direc.: removed …

e38bca9

…DONT_ACCEPT_BLANKLINE (@jnothmam, @reshamas)

Closes scikit-learn#12202 (#wimlds); Removed blank lines in doctest e…

f9352ab

…xample.

weigts in make_classification as sequence not list (#wimlds)

de75f0b

resolved merge

106916f

amueller reviewed Aug 24, 2019

View changes

fix conflicts with upstream/master

ec833ed

CatChenal changed the title ~~Issue 14760~~ Weights parameter of datasets.make_classification changed to sequence from list - Issue 14760 Aug 24, 2019

split if-statement

8e3cb1b

amueller reviewed Aug 24, 2019

View changes

added test test_make_classification_weights_type in test_samples_ge…

776e74e

…nerator.py

CatChenal added 2 commits Aug 26, 2019

fixed flake8 & pylint errors

4b700b8

flake8 err in test file

0c7fec4

NicolasHug reviewed Aug 26, 2019

View changes

CatChenal added 3 commits Aug 28, 2019

Added parametrized tests for weights type.

cac527a

Added untrapped TypeError in samples_generator.py and tests.

6110038

Added untrapped TypeError in samples_generator.py and tests.

7946476

NicolasHug reviewed Sep 3, 2019

View changes

CatChenal added 2 commits Sep 3, 2019

Minor changes as per @NicolasHug

fd2eae6

Corrected assert statement in `test_make_multilabel_classification_…

a6dd8d9

…return_sequences`.

NicolasHug approved these changes Sep 4, 2019

View changes

Added coverage for weiths resizing in `test_make_classification_weigh…

e9f89bf

…ts_array_or_list_ok` as per @NicolasHug.

thomasjpfan reviewed Sep 4, 2019

View changes

Corrected test_make_classification_weights_array_or_list_ok as per @…

0c2a124

…thomasjpfan

thomasjpfan reviewed Sep 5, 2019

View changes

CatChenal added 2 commits Sep 5, 2019

Prettified docstr + updated whats_new/v0.22.rst.

b185474

Fixed rst problem in whats_new/v0.22.rst. :func:package.module.method…

9e7c60b

… is not a linkable target.

thomasjpfan reviewed Sep 5, 2019

View changes

CatChenal added 2 commits Sep 5, 2019

Fixed rst :func: ref as per @thomasjpfan.

d0bf043

Removed organization link in release notes. #WiMLDS`

11a179d

NicolasHug changed the title ~~Weights parameter of datasets.make_classification changed to sequence from list - Issue 14760~~ [MRG] Weights parameter of datasets.make_classification changed to sequence from list - Issue 14760 Sep 9, 2019

CatChenal changed the title ~~[MRG] Weights parameter of datasets.make_classification changed to sequence from list - Issue 14760~~ [MRG] Weights parameter of datasets.make_classification changed to array-like from list only - Issue 14760 Sep 9, 2019

thomasjpfan approved these changes Sep 18, 2019

View changes

thomasjpfan merged commit 8720684 into scikit-learn:master Sep 18, 2019
18 checks passed

	weights : sequence of floats or None (default=None)
	weights : array-like of shape (n_classes,) or (n_classes - 1,), default=None

	if all(weights) and len(weights) == (n_classes - 1):
	if len(weights) == (n_classes - 1):

	if len(weights) == (n_classes - 1):
	if len(weights) == n_classes - 1:

		make_classification(weights=weights)


		def test_make_classification_weights_array_or_list_ok():

	- \|Enhancement\| `make_classification` in :func:`datasets.samples_generator` now
	- \|Enhancement\| :func:`datasets.make_classification` now

	:pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to `WiMLDS <WiMLDS>`.
	:pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to WiMLDS.

scikit-learn / scikit-learn Public

[MRG] Weights parameter of datasets.make_classification changed to array-like from list only - Issue 14760 #14764

[MRG] Weights parameter of datasets.make_classification changed to array-like from list only - Issue 14760 #14764

Conversation

CatChenal commented Aug 24, 2019 • edited

Fixes #14760

What does this implement/fix?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amueller commented Aug 24, 2019

NicolasHug left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NicolasHug left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NicolasHug left a comment

Choose a reason for hiding this comment

thomasjpfan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomasjpfan Sep 5, 2019 • edited

Choose a reason for hiding this comment

thomasjpfan commented Sep 5, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomasjpfan Sep 5, 2019 • edited

Choose a reason for hiding this comment

thomasjpfan Sep 6, 2019 • edited

Choose a reason for hiding this comment

rth Sep 8, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reshamas commented Sep 9, 2019

jnothman commented Sep 9, 2019

reshamas commented Sep 9, 2019

NicolasHug commented Sep 18, 2019

NicolasHug commented Sep 18, 2019

CatChenal commented Sep 18, 2019

CatChenal commented Aug 24, 2019 •

edited

thomasjpfan Sep 5, 2019 •

edited

thomasjpfan commented Sep 5, 2019 •

edited

thomasjpfan Sep 5, 2019 •

edited

thomasjpfan Sep 6, 2019 •

edited

rth Sep 8, 2019 •

edited