Repository navigation
Adding error if columns are all Nan in partial dependence - #2120
freddyaboulton merged 5 commits into
Conversation
| raise ValueError("Partial dependence plots are not supported for Baseline pipelines") | ||
|
|
||
| feature_list = [] | ||
| if isinstance(features, int): |
There was a problem hiding this comment.
Since we only check for int or str we miss the two-way case, which caused #2119
| f"partial dependence cannot be computed: {', '.join(all_nan)}") | ||
|
|
||
|
|
||
| def _raise_value_error_if_mostly_one_value(df, percentile): |
There was a problem hiding this comment.
This is to fix #2119. May be scope-creep but it felt weird to have the all-NaN check work for two-way without fixing this warning. The implementation is pretty simple, we basically use the existing code for the one-way check on every column the user passed in, so I thought it was appropriate.
There was a problem hiding this comment.
Is it weird not using our own datachecks to accomplish this type of thing? This seems very much like a uniqueness and a no variance datacheck.
There was a problem hiding this comment.
Good suggestion! There's definitely overlap in the logic. I'll file a spike issue.
Codecov Report
@@ Coverage Diff @@
## main #2120 +/- ##
=========================================
+ Coverage 100.0% 100.0% +0.1%
=========================================
Files 291 291
Lines 23809 23853 +44
=========================================
+ Hits 23799 23843 +44
Misses 10 10
Continue to review full report at Codecov.
|
| values.append(str(normalized_counts.index[0])) | ||
|
|
||
| if one_value: | ||
| raise ValueError(f"Features {', '.join(one_value)} are mostly one value, ({', '.join(values)}), " |
There was a problem hiding this comment.
nit-pick, but can we put parentheses around the feature names too?
f"Features ({', '.join(one_value)}) are mostlyI think it's cleaner when we have multiple features
There was a problem hiding this comment.
I think this is also another good time to add the raises column to the docstring of the partial dependence function so people can see the linkage of this ValueError to the upper percentile. It took me a second, so I'm sure it'll cause a bit of head scratching.
There was a problem hiding this comment.
Good point @chukarsten . I'll add that now!
| partial_dependence(pipeline, X, features=2, percentiles=(0.01, 0.955), grid_resolution=20) | ||
| with pytest.raises(ValueError, match="Features 'random_col' are mostly one value, \\(1\\), and cannot be"): | ||
| partial_dependence(pipeline, X, features=('A', "random_col"), percentiles=(0.01, 0.955), grid_resolution=20) | ||
| with pytest.raises(ValueError, match="Features 'random_col', 'random_col_2' are mostly one value, \\(1, 1\\), and cannot be"): |
chukarsten
left a comment
There was a problem hiding this comment.
Nice work. Just a suggesting about adding "raises" to the docstring of the part dep function to suggest the relationship of the upper percentile limit to the "mostly a value" ValueError. I also made a comment about using our datachecks to do these seeming datachecks, but I don't know how clean or reasonable that is.
| f"partial dependence cannot be computed: {', '.join(all_nan)}") | ||
|
|
||
|
|
||
| def _raise_value_error_if_mostly_one_value(df, percentile): |
There was a problem hiding this comment.
Is it weird not using our own datachecks to accomplish this type of thing? This seems very much like a uniqueness and a no variance datacheck.
| values.append(str(normalized_counts.index[0])) | ||
|
|
||
| if one_value: | ||
| raise ValueError(f"Features {', '.join(one_value)} are mostly one value, ({', '.join(values)}), " |
There was a problem hiding this comment.
I think this is also another good time to add the raises column to the docstring of the partial dependence function so people can see the linkage of this ValueError to the upper percentile. It took me a second, so I'm sure it'll cause a bit of head scratching.
ParthivNaresh
left a comment
There was a problem hiding this comment.
Overall looks great, excellent addition of helper functions
81d5fd6 to
d053099
Compare
d053099 to
7515f19
Compare
…skEngine`` #1975. - Added optional ``engine`` argument to ``AutoMLSearch`` #1975 - Added a warning about how time series support is still in beta when a user passes in a time series problem to ``AutoMLSearch`` #2118 - Added ``NaturalLanguageNaNDataCheck`` data check #2122 - Added ValueError to ``partial_dependence`` to prevent users from computing partial dependence on columns with all NaNs #2120 - Added standard deviation of cv scores to rankings table #2154 - Fixed ``BalancedClassificationDataCVSplit``, ``BalancedClassificationDataTVSplit``, and ``BalancedClassificationSampler`` to use ``minority:majority`` ratio instead of ``majority:minority`` #2077 - Fixed bug where two-way partial dependence plots with categorical variables were not working correctly #2117 - Fixed bug where ``hyperparameters`` were not displaying properly for pipelines with a list ``component_graph`` and duplicate components #2133 - Fixed bug where ``pipeline_parameters`` argument in ``AutoMLSearch`` was not applied to pipelines passed in as ``allowed_pipelines`` #2133 - Fixed bug where ``AutoMLSearch`` was not applying custom hyperparameters to pipelines with a list ``component_graph`` and duplicate components #2133 - Removed ``hyperparameter_ranges`` from Undersampler and renamed ``balanced_ratio`` to ``sampling_ratio`` for samplers #2113 - Renamed ``TARGET_BINARY_NOT_TWO_EXAMPLES_PER_CLASS`` data check message code to ``TARGET_MULTICLASS_NOT_TWO_EXAMPLES_PER_CLASS`` #2126 - Modified one-way partial dependence plots of categorical features to display data with a bar plot #2117 - Renamed ``score`` column for ``automl.rankings`` as ``mean_cv_score`` #2135 - Fixed ``conf.py`` file #2112 - Added a sentence to the automl user guide stating that our support for time series problems is still in beta. #2118 - Fixed documentation demos #2139 - Update test badge in README to use GitHub Actions #2150 - Fixed ``test_describe_pipeline`` for ``pandas`` ``v1.2.4`` #2129 - Added a GitHub Action for building the conda package #1870 #2148 .. warning:: - Renamed ``balanced_ratio`` to ``sampling_ratio`` for the ``BalancedClassificationDataCVSplit``, ``BalancedClassificationDataTVSplit``, ``BalancedClassficationSampler``, and Undersampler #2113 - Deleted the "errors" key from automl results #1975 - Deleted the ``raise_and_save_error_callback`` and the ``log_and_save_error_callback`` #1975 - Fixed ``BalancedClassificationDataCVSplit``, ``BalancedClassificationDataTVSplit``, and ``BalancedClassificationSampler`` to use minority:majority ratio instead of majority:minority #2077
Pull Request Description
Fixes #2073, #2119
After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of
docs/source/release_notes.rstto include this pull request by adding :pr:123.