Changing Elastic Net Classifier params #2269
Conversation
Codecov Report
@@ Coverage Diff @@
## main #2269 +/- ##
=======================================
Coverage 100.0% 100.0%
=======================================
Files 280 280
Lines 24336 24336
=======================================
Hits 24314 24314
Misses 22 22
Continue to review full report at Codecov.
|
|
Filed an issue to track the SHAP failure here |
bchen1116
changed the title
bchen1116
requested review from
ParthivNaresh,
angela97lin,
chukarsten,
dsherry,
freddyaboulton and
jeremyliweishih
freddyaboulton
left a comment
freddyaboulton
left a comment
There was a problem hiding this comment.
@bchen1116 I think this looks good! Thanks for chasing down the original issue and recommending a very targeted fix. 🎉
I agree that we can tackle the issues with custom objectives you've identified in a separate ticket.
| else: | ||
| training_data, y = X_y_regression | ||
| if "Elastic Net" in estimator.name: | ||
| parameters = {"Elastic Net Classifier": {"alpha": 0.5, "l1_ratio": 0.5, 'n_jobs': 1}} |
There was a problem hiding this comment.
Tracked by #2281 - @bchen1116 can you add a TODO?
| "outputs": [], | ||
| "source": [ | ||
| "X, y = evalml.demos.load_fraud(n_rows=1000)" | ||
| "X, y = evalml.demos.load_fraud(n_rows=5000)" |
There was a problem hiding this comment.
Added n_rows=5000 as stated in the perf test doc here for performance improvement
ParthivNaresh
left a comment
ParthivNaresh
left a comment
There was a problem hiding this comment.
Nice work with the perf tests!
| assert (output_.index == ['Intercept', 'Second', 'Fourth', 'First', 'Third']).all() | ||
| elif estimator.name == 'Elastic Net Regressor': | ||
| assert (output_.index == ['Intercept', 'Second', 'Third', 'Fourth', 'First']).all() | ||
| assert (output_.index == ['Intercept', 'Second', 'Fourth', 'First', 'Third']).all() |
partial fix for #2145.
Docs here
Perf tests here
While this PR does result in better performance in the docs, the main issue that still stands is that the scoring methods we had used (lead_scoring and fraud) allowed a pipeline that always guessed the positive (minority) class to have the best score. To actually solve the issue, it would be useful to tweak the objective functions so that this case would score poorly, otherwise this issue can arise again in the future.
I think that can be a separate PR that can follow some additional discussion. Things we'd need to discuss are: