Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically infer multiclass based off of objective #99

Merged
merged 24 commits into from
Oct 11, 2019

Conversation

jeremyliweishih
Copy link
Contributor

@jeremyliweishih jeremyliweishih commented Sep 23, 2019

This changes the default behavior of multiclass.

If there is a given objective it will do the following:

  • Check if the given objective is compatible with multiclass
  • Set problem_type to MULTICLASS if the given objective is EXCLUSIVELY multiclass
  • Set problem_type to MULTICLASS if compatible

The default objective will be binary precision and if multiclass is set to TRUE it will default to precision_micro.

Also adds a check so users who end up with a mismatch of objective and multiclass will have an easier error to handle.

@codecov
Copy link

codecov bot commented Sep 23, 2019

Codecov Report

Merging #99 into master will increase coverage by 0.23%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #99      +/-   ##
==========================================
+ Coverage   93.49%   93.72%   +0.23%     
==========================================
  Files          52       52              
  Lines        1306     1355      +49     
==========================================
+ Hits         1221     1270      +49     
  Misses         85       85
Impacted Files Coverage Δ
evalml/models/auto_base.py 91.95% <100%> (+0.6%) ⬆️
evalml/objectives/fraud_cost.py 100% <100%> (ø) ⬆️
evalml/tests/automl_tests/test_autoclassifier.py 100% <100%> (ø) ⬆️
evalml/problem_types/problem_types.py 100% <100%> (ø) ⬆️
evalml/objectives/lead_scoring.py 100% <100%> (ø) ⬆️
evalml/models/auto_classifier.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c671859...966d5fa. Read the comment docs.

@jeremyliweishih jeremyliweishih changed the title WIP: Added infer base off objective Added infer base off objective Sep 25, 2019
@jeremyliweishih jeremyliweishih changed the title Added infer base off objective Automatically infer multiclass based off of objective Sep 25, 2019
Copy link
Contributor

@kmax12 kmax12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic seems good. just some comments on making code a bit cleaner

evalml/models/auto_base.py Outdated Show resolved Hide resolved
evalml/models/auto_base.py Outdated Show resolved Hide resolved
evalml/models/auto_classifier.py Outdated Show resolved Hide resolved
evalml/models/auto_classifier.py Outdated Show resolved Hide resolved
evalml/models/auto_base.py Show resolved Hide resolved
Copy link
Contributor

@kmax12 kmax12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jeremyliweishih jeremyliweishih merged commit 4a819b8 into master Oct 11, 2019
@angela97lin angela97lin mentioned this pull request Oct 29, 2019
@dsherry dsherry deleted the auto-infer branch May 26, 2020 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants