Change undersampler to use minority:majority ratio #2077
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #2077 +/- ##
=========================================
+ Coverage 100.0% 100.0% +0.1%
=========================================
Files 288 288
Lines 23419 23428 +9
=========================================
+ Hits 23409 23418 +9
Misses 10 10
Continue to review full report at Codecov.
|
bchen1116
requested review from
ParthivNaresh,
chukarsten,
dsherry,
freddyaboulton and
jeremyliweishih
dsherry
suggested changes
Apr 5, 2021
Contributor
dsherry
left a comment
dsherry
left a comment
There was a problem hiding this comment.
Looking close! I think I found one minor bug, and I had a couple test suggestions.
evalml/pipelines/components/transformers/samplers/undersampler.py
Outdated
Show resolved
Hide resolved
evalml/preprocessing/data_splitters/balanced_classification_sampler.py
Outdated
Show resolved
Hide resolved
evalml/preprocessing/data_splitters/balanced_classification_sampler.py
Outdated
Show resolved
Hide resolved
| X2 = X.loc[indices] | ||
| y2 = y.loc[indices] | ||
| if balanced_ratio >= 3: | ||
| if balanced_ratio <= .3: |
Contributor
There was a problem hiding this comment.
This is because the default balanced_ratio is 0.25? Could you say if balanced_ratio <= 0.25 for clarity?
(Its probably more trouble than its worth but you could also use introspection to retrieve the default balanced_ratio value from the method signature 😆 )
Contributor
Author
There was a problem hiding this comment.
@dsherry We choose 0.3 since the targets have a class ratio of 1/3 for the minority/majority balance. In scenarios where we consider anything less than 1/3 as balanced (for the sampler), we expect the data to not be undersampled. For instance, if we consider class ratios down to 1/5 as balanced, then this dataset shouldn't be sampled.
evalml/tests/preprocessing_tests/test_balanced_classification_sampler.py
Show resolved
Hide resolved
| X2 = X.loc[indices] | ||
| y2 = y.loc[indices] | ||
| if balanced_ratio >= 6: | ||
| if balanced_ratio < .2: |
Contributor
There was a problem hiding this comment.
Same as before, please change to 0.25 for clarity, and I see one more similar usage below.
Contributor
Author
There was a problem hiding this comment.
Similar reasoning as before!
bchen1116
commented
Apr 6, 2021
chukarsten
added a commit
that referenced
this pull request
Apr 20, 2021
…skEngine`` #1975. - Added optional ``engine`` argument to ``AutoMLSearch`` #1975 - Added a warning about how time series support is still in beta when a user passes in a time series problem to ``AutoMLSearch`` #2118 - Added ``NaturalLanguageNaNDataCheck`` data check #2122 - Added ValueError to ``partial_dependence`` to prevent users from computing partial dependence on columns with all NaNs #2120 - Added standard deviation of cv scores to rankings table #2154 - Fixed ``BalancedClassificationDataCVSplit``, ``BalancedClassificationDataTVSplit``, and ``BalancedClassificationSampler`` to use ``minority:majority`` ratio instead of ``majority:minority`` #2077 - Fixed bug where two-way partial dependence plots with categorical variables were not working correctly #2117 - Fixed bug where ``hyperparameters`` were not displaying properly for pipelines with a list ``component_graph`` and duplicate components #2133 - Fixed bug where ``pipeline_parameters`` argument in ``AutoMLSearch`` was not applied to pipelines passed in as ``allowed_pipelines`` #2133 - Fixed bug where ``AutoMLSearch`` was not applying custom hyperparameters to pipelines with a list ``component_graph`` and duplicate components #2133 - Removed ``hyperparameter_ranges`` from Undersampler and renamed ``balanced_ratio`` to ``sampling_ratio`` for samplers #2113 - Renamed ``TARGET_BINARY_NOT_TWO_EXAMPLES_PER_CLASS`` data check message code to ``TARGET_MULTICLASS_NOT_TWO_EXAMPLES_PER_CLASS`` #2126 - Modified one-way partial dependence plots of categorical features to display data with a bar plot #2117 - Renamed ``score`` column for ``automl.rankings`` as ``mean_cv_score`` #2135 - Fixed ``conf.py`` file #2112 - Added a sentence to the automl user guide stating that our support for time series problems is still in beta. #2118 - Fixed documentation demos #2139 - Update test badge in README to use GitHub Actions #2150 - Fixed ``test_describe_pipeline`` for ``pandas`` ``v1.2.4`` #2129 - Added a GitHub Action for building the conda package #1870 #2148 .. warning:: - Renamed ``balanced_ratio`` to ``sampling_ratio`` for the ``BalancedClassificationDataCVSplit``, ``BalancedClassificationDataTVSplit``, ``BalancedClassficationSampler``, and Undersampler #2113 - Deleted the "errors" key from automl results #1975 - Deleted the ``raise_and_save_error_callback`` and the ``log_and_save_error_callback`` #1975 - Fixed ``BalancedClassificationDataCVSplit``, ``BalancedClassificationDataTVSplit``, and ``BalancedClassificationSampler`` to use minority:majority ratio instead of majority:minority #2077
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix #2076