[MRG] text vectorizers should raise warnings when user params will be unused #14602

getgaurav2 · 2019-08-08T06:10:23Z

Reference Issues/PRs

#14580

What does this implement/fix? Explain your changes.

Add Warning and test cases for unused parameters for text vectorizers.

Add all warnings in a new method - _warn_for_unused_params() and call this method in the fit() method for all vectorizers .
Add Unit test case for all warning checks .

Any other comments?

# Conflicts: # sklearn/compose/_column_transformer.py

…lyzer pair

getgaurav2 · 2019-08-08T08:33:26Z

@jnothman , @rth - can you pls. give any feedback on the test case ?
I can add the other text vectorizers in the same function and iterate over the objects of each vectorizers and check different warnings msg for different parameter combinations
OR
I can create one unit test function per vectorizer per warning .

sklearn/feature_extraction/text.py

sklearn/feature_extraction/tests/test_text.py

Co-Authored-By: Roman Yurchak <rth.yurchak@gmail.com>

jnothman

Looking good

getgaurav2 · 2019-08-09T00:13:31Z

getgaurav2

Adding only CountVectorizer for now . Will add others as I add warnings for them in the text.py file.

…failed.

getgaurav2 · 2019-08-09T00:30:02Z

@jnothman , @rth - can you please check why some of the checks are failing for this commit.
Here is the log :
scikit-learn.scikit-learn (Linux py35_conda_openblas)
PackagesNotFoundError: The following packages are not available from current channels:

numpy=1.11.0
scipy=0.17.0
matplotlib=1.5.1
pillow=4.0.0

Linux pylatest_conda_mkl_pandas
~/work/tmp_folder ~/work/1/s
build_tools/azure/test_pytest_soft_dependency.sh: line 21: coverage: command not found

thomasjpfan · 2019-08-10T02:31:22Z

It is a unrelated issue.

It should be fixed when #14619 get merged.

rth

LGTM, apart for the two comments below.

sklearn/feature_extraction/tests/test_text.py

jnothman · 2019-08-15T11:43:08Z

HashingVectorizer doesn't do much in fit. I suspect we should change it to call _build_analyzer just as a smoke test.

jnothman

Thanks

sklearn/feature_extraction/text.py

2. Adding condition for self.analyzer != 'word' or callable(self.analyzer) in build_analyzer

jnothman · 2019-08-18T00:52:24Z

Tests are failing . Let us know if you need help

…nto 14580

getgaurav2 · 2019-08-18T05:51:31Z

Tests are failing . Let us know if you need help

Thanks. A different unit test test_callable_analyzer_change_behavior seems to be failing ...I'll check the reason and get back to you .

getgaurav2 · 2019-08-24T15:00:47Z

Tests are failing . Let us know if you need help

Thanks. A different unit test test_callable_analyzer_change_behavior seems to be failing ...I'll check the reason and get back to you .

@jnothman - Could you help me understand the reason for adding assert len(records) == 1 in test_callable_analyzer_change_behavior . This condition is failing for HashingVectorizer ( screen shot below). It passes for CountVectorizer and HashingVectorizer . I have commented this condition in the test case for now but will add it back once I understand it better.

The difference b/w the fit() method for HashingVectorizer () vs. the others is calling build_analyzer() . https://github.com/scikit-learn/scikit-learn/pull/14602#issuecomment-521613247

jnothman · 2019-08-24T22:59:27Z

It's saying that HashingVectorizer is now issuing the ChangedBehaviorWarning twice. It should only issue it once. By putting build_analyzer in its fit, it might now be repeating that warning.

getgaurav2 · 2019-08-25T00:37:19Z

It's saying that HashingVectorizer is now issuing the ChangedBehaviorWarning twice. It should only issue it once. By putting build_analyzer in its fit, it might now be repeating that warning.

Yes , build_analyzer () is now being called inside fit() as well as transform() methods which is likely causing the double warnings . Looking into it now .
@jnothman - Do you think it may be easier to go back to the original approach and keep the analyzer != ‘word’ warnings outside of build_ analyzer ?
#14602 (comment)

getgaurav2 · 2019-08-28T11:59:01Z

@jnothman - just wanted to make sure that you are getting a notification for the comment above.

jnothman

Otherwise LGTM. (Though I've not checked that the set of parameter pairs is complete.)

Please add an Enhancement entry to the change log at doc/whats_new/v0.22.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:

sklearn/feature_extraction/tests/test_text.py

sklearn/feature_extraction/text.py

jnothman · 2019-09-04T10:33:13Z

Assuming you also think this is ready for merge, please change WIP -> MRG

…unused_params And remove blank lines

jnothman · 2019-09-05T00:31:00Z

Please add a note in what's new as requested above. Thanks!

getgaurav2 · 2019-09-05T00:52:06Z

Please add a note in what's new as requested above. Thanks!

Yup ..Was just working on it. Added now .

jnothman · 2019-09-05T06:28:12Z

doc/whats_new/v0.22.rst

@@ -146,6 +146,14 @@ Changelog
 :mod:`sklearn.feature_extraction`
 .................................

+- |Enhancement| :func:`feature_extraction.text._warn_for_unused_params` will


Please describe changes only in terms of public API: "a warning is now raised if a parameter choice means that another parameter will be unused" or something more clear than that.

Got it ..Can you please check if the latest commit makes more sense .

doc/whats_new/v0.22.rst

jnothman · 2019-09-06T01:08:21Z

Makes more sense. We might yet find clearer words before release, but as far as I'm concerned we can merge. Thanks, @getgaurav2!

Gaurav Chawla added 15 commits Jul 4, 2019

Adding link to User guide for make_column_transformer

2662657

Describe the arguments for *transformers.

a57f617

Merge remote-tracking branch 'upstream/master'

799408d

# Conflicts: # sklearn/compose/_column_transformer.py

#14580 - Add first warning for CountVectorizer for stop_words and ana…

7262214

…lyzer pair

removing spaces

958ccaa

changing message

e91c0df

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn

1251dca

#14580 - Add Warning for analyzer and stop word pair

439e59c

modifying warning text

15a58d2

moving the code logic to fit

5ddb9da

bracket

5491fde

flake8 changes

dc61048

Adding Unit test case for Stop Word and analyzer != 'word

0ef396c

flake8 changes

67168ea

more flake8

0c6f5cd

rth reviewed Aug 8, 2019

View changes

getgaurav2 and others added 3 commits Aug 8, 2019

Update sklearn/feature_extraction/tests/test_text.py

5cc792b

Co-Authored-By: Roman Yurchak <rth.yurchak@gmail.com>

incorporating suggestions from @rth

2f0f572

flake 8 suggestions

0131118

jnothman reviewed Aug 8, 2019

View changes

Using pytest.warns and parameterizing the test case .

1321a32

getgaurav2 reviewed Aug 9, 2019

View changes

Adding minor changes to trigger another build since a few test cases …

e8a338c

…failed.

Gaurav Chawla added 2 commits Aug 10, 2019

Adding Warning and Unit test for analyzer and tokenizer pair

28ac365

Adding warning and unit test for tokenizer and token-pattern pair

b0e1431

rth approved these changes Aug 10, 2019

View changes

sklearn/feature_extraction/tests/test_text.py Outdated Show resolved Hide resolved

sklearn/feature_extraction/tests/test_text.py Outdated Show resolved Hide resolved

calling build_analyzer inside fit() method for Hashing vectorizer

7564966

jnothman reviewed Aug 16, 2019

View changes

sklearn/feature_extraction/text.py Outdated Show resolved Hide resolved

sklearn/feature_extraction/text.py Outdated Show resolved Hide resolved

1. Further parameterizing the unit test function

872be9f

2. Adding condition for self.analyzer != 'word' or callable(self.analyzer) in build_analyzer

Gaurav Chawla added 3 commits Aug 18, 2019

Merge branch 'master' of https://github.com/getgaurav2/scikit-learn i…

002b1b3

…nto 14580

flake8 changes

4da39b3

resolving merge conflicts

93e4e6f

Gaurav Chawla added 2 commits Aug 24, 2019

removing an extra warning added by mistake.

7e8219e

commenting assert len(records) == 1 for the time

26d8ae0

Taking the analyzer!= word validations outside of build_analyzer

4c1e87c

jnothman approved these changes Sep 4, 2019

View changes

Rename the warning function from _missing_param_warning to _warn_for_…

42fba5e

…unused_params And remove blank lines

getgaurav2 changed the title ~~[WIP] text vectorizers should raise warnings when user params will be unused~~ [MRG] text vectorizers should raise warnings when user params will be unused Sep 4, 2019

Adding enhancement section for this pull request

0268483

minor fixes

6897fdf

jnothman reviewed Sep 5, 2019

View changes

modifying language

706405a

jnothman merged commit 96ef6b8 into scikit-learn:master Sep 6, 2019
17 checks passed

sameshl mentioned this pull request Sep 28, 2019

MNT raise warning in CountVectorizer #14601

Closed

rth mentioned this pull request Sep 28, 2019

text vectorizers should raise warnings when user params will be unused #14580

Closed

scikit-learn / scikit-learn Public

[MRG] text vectorizers should raise warnings when user params will be unused #14602

[MRG] text vectorizers should raise warnings when user params will be unused #14602

getgaurav2 commented Aug 8, 2019 •

edited

getgaurav2 commented Aug 8, 2019 •

edited

jnothman left a comment

getgaurav2 commented Aug 9, 2019

getgaurav2 left a comment

getgaurav2 commented Aug 9, 2019 •

edited

thomasjpfan commented Aug 10, 2019

rth left a comment

jnothman commented Aug 15, 2019

jnothman left a comment

jnothman commented Aug 18, 2019

getgaurav2 commented Aug 18, 2019 •

edited

getgaurav2 commented Aug 24, 2019 •

edited

jnothman commented Aug 24, 2019

getgaurav2 commented Aug 25, 2019 •

edited

getgaurav2 commented Aug 28, 2019

jnothman left a comment

jnothman commented Sep 4, 2019

jnothman commented Sep 5, 2019

getgaurav2 commented Sep 5, 2019

jnothman Sep 5, 2019

getgaurav2 Sep 5, 2019

jnothman commented Sep 6, 2019

scikit-learn / scikit-learn Public

[MRG] text vectorizers should raise warnings when user params will be unused #14602

[MRG] text vectorizers should raise warnings when user params will be unused #14602

Conversation

getgaurav2 commented Aug 8, 2019 • edited

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

getgaurav2 commented Aug 8, 2019 • edited

jnothman left a comment

getgaurav2 commented Aug 9, 2019

getgaurav2 left a comment

getgaurav2 commented Aug 9, 2019 • edited

thomasjpfan commented Aug 10, 2019

rth left a comment

jnothman commented Aug 15, 2019

jnothman left a comment

jnothman commented Aug 18, 2019

getgaurav2 commented Aug 18, 2019 • edited

getgaurav2 commented Aug 24, 2019 • edited

jnothman commented Aug 24, 2019

getgaurav2 commented Aug 25, 2019 • edited

getgaurav2 commented Aug 28, 2019

jnothman left a comment

jnothman commented Sep 4, 2019

jnothman commented Sep 5, 2019

getgaurav2 commented Sep 5, 2019

jnothman Sep 5, 2019

Choose a reason for hiding this comment

getgaurav2 Sep 5, 2019

Choose a reason for hiding this comment

jnothman commented Sep 6, 2019

getgaurav2 commented Aug 8, 2019 •

edited

getgaurav2 commented Aug 8, 2019 •

edited

getgaurav2 commented Aug 9, 2019 •

edited

getgaurav2 commented Aug 18, 2019 •

edited

getgaurav2 commented Aug 24, 2019 •

edited

getgaurav2 commented Aug 25, 2019 •

edited