[MRG] Weights parameter of datasets.make_classification changed to array-like from list only - Issue 14760

[CatChenal]

Fixes #14760

What does this implement/fix?

The weights parameter can be a list or array, not just a list, e.g.
" weights is array-like or None " in docstring.

[amueller]

maybe weights is not None would be more clear?

[amueller]

merge issues here

[amueller]

the idiomatic python way is weights is not None.
I would use a single if and and. You can put the whole condition in parenthesis and make it multi-line

[CatChenal]

ok.

[amueller]

this change seems unrelated

[NicolasHug]

ping @CatChenal can you please revert these changes

[amueller]

Please add a non-regression test that would fail at master but pass in this PR.

[NicolasHug]

Thanks for the PR @CatChenal ,

made a few comments

[NicolasHug]

We call this an array-like.

Suggested change

	weights : sequence of floats or None (default=None)
	weights : array-like of shape (n_classes,) or (n_classes - 1,), default=None

[NicolasHug]

shouldn't an error be raised if all(weights) is false?

[NicolasHug]

Suggested change

	if all(weights) and len(weights) == (n_classes - 1):
	if len(weights) == (n_classes - 1):

[NicolasHug]

That's not you, but that line is useless ;)

[CatChenal]

Which line?
Line 175 resizes the (n-classes - 1) array with the missing weight, so it makes sense.
Line 178 recalculate to last position of weights according to values set on line 177;
Line 178 is the useless, no?

[NicolasHug]

weights = [1.0 / n_classes] * n_classes
weights[-1] = 1.0 - sum(weights[:-1])  # <-- this one

[NicolasHug]

We like to parametrize these kind of tests. You can look for some inspiration in e.g. this test

Let us know if you need any help

[CatChenal]

None of the tests in test_samples_generator.py are parametrized. Do you want me to parametrize all of them or just the tests for make_classification()?

[CatChenal]

Oops, found one: test_make_blobs_n_samples_centers_none()

[NicolasHug]

I was just suggesting to parametrize the test you wrote

[CatChenal]

Good...
I just found out one wrong way to do it:

@pytest.mark.parametrize(
    'params, err_msg',
    [({'weights': 0}, "object of type 'int' has no len()"),
     ({'weights': -1}, "object of type 'int' has no len()"),
     ({'weights': []}, "Weights specified but incompatible with number of classes."),
     ({'weights': [.25,.75,.1]}, "Weights specified but incompatible with number of classes."),
     ({'weights': np.array([])},"Weights specified but incompatible with number of classes."),
     ({'weights': np.array([.25,.75,.1])},"Weights specified but incompatible with number of classes.")]
)
def test_make_classification_weights_type(params, err_msg):
    make = partial(make_classification,
                   n_samples=100,
                   n_features=20,
                   n_informative=2,
                   n_redundant=2,
                   n_repeated=0,
                   n_classes=2,
                   n_clusters_per_class=2,
                   flip_y=0.01,
                   class_sep=1.0,
                   hypercube=True,
                   shift=0.0,
                   scale=1.0,
                   shuffle=True,
                   random_state=0)
    
    for i in range(len(params)):
        with pytest.raises(ValueError, match=err_msg[i]):
            make(weights=params[i]['weights'])

The first problem is that the mark.parametrize statement is incorrect: the weights in the partial functions are not split and I have not found out how to fix it yet.
The other problem is likely the iteration of the context manager (should not be needed...).
Thanks for pointing me in the right direction.

[NicolasHug]

You're almost there. Here is a basic example:

@pytest.mark.parametrize(
    'weights, err_msg',
    [
        ([1, 2, 3], "incompatible with number of classes"),
        # add other test cases here
    ]
)
def test_make_classification_weights_type(weights, err_msg):

    with pytest.raises(ValueError, match=err_msg):
        make_classification(weights=weights)

[NicolasHug]

Thanks @CatChenal , I made a few more comments but mostly looks good. Could you also please add a very simple test that makes sure passing e.g. [1, 2, 3] gives the same result as passing np.array([1, 2. 3]). Thanks!

[NicolasHug]

ping @CatChenal can you please revert these changes

[NicolasHug]

Please avoid unrelated changes

[CatChenal]

ok

[NicolasHug]

same here

[CatChenal]

ok

[NicolasHug]

same here

[CatChenal]

ok

[NicolasHug]

I don't think we need a specific check for int (else it means we would need specific checks for pretty much every type). I guess a safe way is to convert the weights to a numpy array. You can then just check the length as you do below, and use np.sum everywhere.

[NicolasHug]

Suggested change

	if len(weights) == (n_classes - 1):
	if len(weights) == n_classes - 1:

[CatChenal]

ok

[NicolasHug]

Minor comment about test coverage, but LGTM anyway. Thanks @CatChenal !

[NicolasHug]

That part isn't covered by the tests. I think you can cover it easily by setting n_classes=3 in test_make_classification_weights_array_or_list_ok.

[thomasjpfan]

Thank you @CatChenal for working on this!

[thomasjpfan]

X1.all() returns True if X1 is all non-zero. Is this assertion to do the following:

assert_almost_equal(X1, X2)
assert_almost_equal(y1, y2)

[thomasjpfan]

This can be parametrized:

@pytest.mark.parametrize("kwargs", [{}, {"n_classes": 3, "n_informative": 3}])
def test_make_classification_weights_array_or_list_ok(kwargs):
    X1, y1 = make_classification(weights=[.1, .9],
                                 random_state=0, **kwargs)
    X2, y2 = make_classification(weights=np.array([.1, .9]),
                                 random_state=0, **kwargs)
    assert_almost_equal(X1, X2)
    assert_almost_equal(y1, y2)

[CatChenal]

Thank you!

[thomasjpfan]

Currently this is not rendered nicely.

To render nicely:

Suggested change

	weights : array-like of shape (n_classes,) or (n_classes - 1,),
	weights : array-like of shape (n_classes,) or (n_classes - 1,),\

[CatChenal]

Thanks, @thomasjpfan.
Would you please document how you reached that end-point to verify the rendering? My doc tree does not have a /modules/generated/ path.

[thomasjpfan]

When you build the html documentation using these instructions, there will be a new folder: doc/_build which contains doc/_build/html/stable/index.html which is the landing page of the scikit-learn. From there you can navigate to the make_classification docs by going to the API page.

[thomasjpfan]

Please add Enhancement entry to the change log at doc/whats_new/v0.22.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:.

[thomasjpfan]

Suggested change

	- |Enhancement| `make_classification` in :func:`datasets.samples_generator` now
	- |Enhancement| :func:`datasets.make_classification` now

[CatChenal]

Of course!

[thomasjpfan]

Suggested change

	:pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to `WiMLDS <WiMLDS>`.
	:pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to *WiMLDS*.

[CatChenal]

Certainly not. That would downgrade @WiMLDS's contributions to Opensource & Scikit-learn in particular.

[thomasjpfan]

This would link you and @WiMLDS

- |Enhancement| :func:`datasets.make_classification` now accepts array-like
  `weights` parameter, i.e. list or numpy.array, instead of list only.
  :pr:`14764` by :user:`Cat Chenal <CatChenal>`, with thanks to
  :user:`WiMLDS <WiMLDS>`.

[thomasjpfan]

Certainly not. That would downgrade @WiMLDS's contributions to Opensource & Scikit-learn in particular.

Sorry, I misunderstood the intend of the string. I can see now you were trying to link to the organization on github. The above snippet should correctly link to their organization.

[rth]

I'm not sure about this, we usually acknowledge individuals not organizations in release notes. For funding organizations are typically mentioned in https://scikit-learn.org/stable/about.html#funding). I think we could maybe add a section for WiMLDS and similar partner non-profit organizations there? The problem with acknowledgements of organizations in release notes is that most contributions have some sort of organization behind it ( Numfocus, conference where the sprint happened, company who allowed its employee to contribute during work time, etc), and then deciding to acknowledge some but not others is tricky.

[CatChenal]

Thanks for the information. The mention is then out of place in the release notes. I will remove it & add #WiMLDS in the final commit.

[reshamas]

@NicolasHug
Does this title need to include "MRG" ?

cc: @kellycarmody

[jnothman]

Is WiMLDS being listed here as a sponsor, or rather a way that the contributor was able to learn to contribute? I like the WiMLDS mention in the change log.

[reshamas]

@jnothman

Is WiMLDS being listed here as a sponsor, or rather a way that the contributor was able to learn to contribute? I like the WiMLDS mention in the change log.

WiMLDS contributed in the following ways:

• organized the event: provided a way for people to contribute
• sponsored the event: funding
• sprint contributors
• donated to NumFOCUS, for scikit-learn

Any way that is acknowledged would be cool.

[NicolasHug]

@thomasjpfan @rth comments were addressed it seems, let's merge?

[NicolasHug]

Thanks @CatChenal !!

[CatChenal]

Thank you @NicolasHug and @thomasjpfan!