Make utils to handle the logic for threshold tuning objective and resplitting data #3888
Conversation
| @@ -261,3 +262,62 @@ def get_pipelines_from_component_graphs( | |||
| ), | |||
| ) | |||
| return created_pipelines | |||
|
|
|||
|
|
|||
| def get_threshold_tuning_info(automl_config, pipeline): | |||
There was a problem hiding this comment.
I stuck these in automl utils instead of putting them in the engine_base file/making a new engine utils file, because part of their use is in determining after automl search whether pipelines were trained on the full data. So since they can be used in the wider context of automl, I thought it made sense to put them there. If anyone has any strong feelings to the contrary, definitely let me know!
| @@ -103,7 +103,7 @@ def test_train_and_score_pipelines_error( | |||
| assert "yeet" in caplog.text | |||
|
|
|||
|
|
|||
| @patch("evalml.automl.engine.engine_base.split_data") | |||
| @patch("evalml.automl.utils.split_data") | |||
There was a problem hiding this comment.
The logic isn't changing, so I didn't add any tests that specifically test these utils on their own, but I'd be open to adding some if that's a necessary part of pulling this logic out into public utils
tamargrey
force-pushed
the
make-resplit-utils
branch
from
ec96078 to
4ab5c33
Compare
Codecov Report
@@ Coverage Diff @@
## main #3888 +/- ##
=======================================
+ Coverage 99.7% 99.7% +0.1%
=======================================
Files 346 346
Lines 36352 36358 +6
=======================================
+ Hits 36221 36227 +6
Misses 131 131
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
tamargrey
requested review from
jeremyliweishih,
christopherbunn and
eccabay
docs/source/release_notes.rst
Outdated
| @@ -2,6 +2,7 @@ Release Notes | |||
| ------------- | |||
| **Future Releases** | |||
| * Enhancements | |||
| * Make utils to handle the logic for threshold tuning objective and resplitting data :pr:`3888` | |||
There was a problem hiding this comment.
I have decided past tense release notes are the hill I'm going to die on. Can we do make --> made? Or even really "added" instead.
There was a problem hiding this comment.
ah damn! I had noticed that and had been doing past tense to fit in. I'll make the change!
For what it's worth, my commit message (and release note since I mostly treat them the same) writing style comes from this article that I was forced asked to read at my first internship. The thing that made the biggest impression on me was this section
There was a problem hiding this comment.
okay, I see the internet is kind of split between what tense release notes should be in, but more of them say past tense than present tense.
There was a problem hiding this comment.
Yeah, my personal ethos has always been past tense for release notes and present tense for commits, and that seems to be what we generally follow. Super interesting article though, lots to think about there!
tamargrey
force-pushed
the
make-resplit-utils
branch
from
bac699c to
9f7d393
Compare
closes #3885
closes #3863
Allows us to determine for pipelines produced by automl search whether the full training data was used to train them.