Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create binary and multiclass objective classes #504

Merged
merged 145 commits into from Apr 3, 2020

Conversation

angela97lin
Copy link
Contributor

@angela97lin angela97lin commented Mar 17, 2020

Continuation of #419 which was closed after a force push 馃槄(Closes #504, #488)

As part of #346, we need to split our objectives into binary and multiclass objectives.

angela97lin and others added 30 commits Feb 19, 2020
* Update latest dependencies
* adding functionality and test

* changelog and adding boolean param
* Add a whitelist for update_deps check

* Remove from expected

* Update deps

* Changelog
evalml/objectives/objective_base.py Outdated Show resolved Hide resolved

def decision_function(self, y_predicted, extra_cols, threshold):
"""Determine if transaction is fraud given predicted probabilities, dataframe with transaction amount, and threshold
def decision_function(self, ypred_proba, threshold=0.0, X=None):
Copy link
Collaborator

@dsherry dsherry Apr 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great.

Thinking out loud, I wonder if in the future we'll want to have the threshold default to None here and throw an error, to indicate to users that they must always decide on a threshold.

evalml/objectives/fraud_cost.py Outdated Show resolved Hide resolved
evalml/objectives/lead_scoring.py Outdated Show resolved Hide resolved
evalml/objectives/objective_base.py Outdated Show resolved Hide resolved
evalml/objectives/objective_base.py Outdated Show resolved Hide resolved
evalml/objectives/utils.py Outdated Show resolved Hide resolved
dsherry
dsherry approved these changes Apr 3, 2020
Copy link
Collaborator

@dsherry dsherry left a comment

LGTM!! Nice work shepherding this through! This is huge :D and very close to getting into master!

The one final thing I'd like before you merge is that let's delete component_graph = [] from ClassificationPipeline as that could cause bugs. Other than that, all the important comments can go be addressed in a separate PR or filed as an issue.

I haven't finished going through the tests, but since this is merging to a feature branch, let's merge now and address test comments in a separate PR too.

@dsherry
Copy link
Collaborator

dsherry commented Apr 3, 2020

Ok cool I see your latest updates now. Good to merge from my perspective! 馃コ

@angela97lin angela97lin merged commit 55d737a into improved_objectives Apr 3, 2020
2 checks passed
angela97lin added a commit that referenced this pull request Apr 13, 2020
* Objectives API: Create new binary / multiclass pipeline classes and remove objectives from pipeline classes (#405)

* Objectives API: Remove ROC and confusion matrix as objectives (#422)

* Change `score` output to return one dictionary (#429)

* Create binary and multiclass objective classes  (#504)

* Update dependencies  (#412)

* Hide features with zero importance in plot by default (#413)

* Update dependencies check: package whitelist (#417)

* Add fixes necessary for docs to build for improved objectives project (#605)

* Remove calculating plot metrics from AutoML  (#615)
@dsherry dsherry deleted the 373_objective_classes branch Oct 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants