Skip to content

Flatten data check action validate API#3244

Merged
angela97lin merged 19 commits intoupdate_data_check_action_APIfrom
flatten_data_check_action_API
Jan 18, 2022
Merged

Flatten data check action validate API#3244
angela97lin merged 19 commits intoupdate_data_check_action_APIfrom
flatten_data_check_action_API

Conversation

@angela97lin
Copy link
Contributor

@angela97lin angela97lin commented Jan 13, 2022 •
edited
Loading

Closes #3243. This is being merged into my DC branch, not main!

Lots of lines touched but not much logic.

@angela97lin angela97lin self-assigned this Jan 13, 2022
@codecov
Copy link

codecov bot commented Jan 13, 2022 •
edited
Loading

Codecov Report

Merging #3244 (05fa98d) into update_data_check_action_API (556c437) will decrease coverage by 0.1%.
The diff coverage is 99.8%.

Impacted file tree graph

@@                      Coverage Diff                       @@
##           update_data_check_action_API   #3244     +/-   ##
==============================================================
- Coverage                          99.8%   99.8%   -0.0%     
==============================================================
  Files                               326     326             
  Lines                             31589   31586      -3     
==============================================================
- Hits                              31498   31495      -3     
  Misses                               91      91
Impacted Files Coverage Δ
evalml/data_checks/data_check.py 100.0% <ø> (ø)
...tests/data_checks_tests/test_data_check_message.py 100.0% <ø> (ø)
..._tests/test_data_checks_and_actions_integration.py 99.0% <91.7%> (-1.0%) ⬇️
evalml/automl/automl_search.py 99.9% <100.0%> (+0.1%) ⬆️
evalml/data_checks/class_imbalance_data_check.py 100.0% <100.0%> (ø)
evalml/data_checks/data_check_message.py 100.0% <100.0%> (ø)
evalml/data_checks/data_checks.py 100.0% <100.0%> (ø)
evalml/data_checks/datetime_format_data_check.py 100.0% <100.0%> (ø)
evalml/data_checks/datetime_nan_data_check.py 100.0% <100.0%> (ø)
evalml/data_checks/id_columns_data_check.py 100.0% <100.0%> (ø)
... and 32 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 556c437...05fa98d. Read the comment docs.

@angela97lin angela97lin changed the title [Spike] Flatten data check action validate API Flatten data check action validate API Jan 18, 2022
@angela97lin angela97lin marked this pull request as ready for review January 18, 2022 08:47
eccabay
eccabay reviewed Jan 18, 2022
View reviewed changes
Copy link
Contributor

@eccabay eccabay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a big fan of this simplification!

freddyaboulton
freddyaboulton approved these changes Jan 18, 2022
View reviewed changes
Copy link
Contributor

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angela97lin I agree the new api format is easier to understand by treating actions/warnings/errors the same. If the team is ok with the breaking change then this looks good to me!

ParthivNaresh
ParthivNaresh approved these changes Jan 18, 2022
View reviewed changes
Copy link
Contributor

@ParthivNaresh ParthivNaresh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome I love it!

@angela97lin angela97lin merged commit 37abbc1 into update_data_check_action_API Jan 18, 2022
@angela97lin angela97lin deleted the flatten_data_check_action_API branch January 18, 2022 19:05
angela97lin added a commit that referenced this pull request Jan 25, 2022
@angela97lin
Update data check actions API to return options instead of actions an…
9e36684 
…d add functionality to suggest and take action on columns with null values (#3182)

* Update `validate()` API (#3142)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* fix doctests

* fix ts splitting test and impl

* fix more ts splitting tests

* fix doctest for ts data check

* release notes

* update target leakage data check docstring for consistency

* doctest linting and cleanup

* linting

* fix merging main issues

* Update `validate()` API to use `DataCheckActionOption` instead of `DataCheckAction` (#3152)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* add more tests

* Rename `HighlyNullDataCheck` to `NullDataCheck` and update data check to return impute action for non-highly null columns. (#3197)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* init rename

* retrigger

* revert release nots

* add new logic for detecting null cols

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix test and iteration for null data check

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* oops fix merge

* add more tests

* fix null data check tests

* update to use per column strategy and fix tests

* fix tests for data checks

* fix tests and doctests

* release notes

* fix release notes

* oops update action code

* oops fix test

* update wording of messages

* move logic out of dict

* update mode to most_frequent for impute strategies

* oops fix linting and doctests

* Flatten data check action ``validate`` API (#3244)

* init

* more cleanup

* begin to clean up tests

* fix more tests

* updating naming to action_options and fixing more tests

* fix no variance

* fix another test

* fixing automl tests

* oops actually fix automl tests

* fix the other automl tests

* fixing notebook

* fix data check tests

* fix doctests and docs

* integration tests and cleanup

* cleanup based on comments

* linting

* clean up notebook and tests

* Update `make_pipeline_from_actions` to handle null column imputation (#3237)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* init rename

* retrigger

* revert release nots

* add new logic for detecting null cols

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix test and iteration for null data check

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* oops fix merge

* add more tests

* fix null data check tests

* update to use per column strategy and fix tests

* fix tests for data checks

* fix tests and doctests

* release notes

* fix release notes

* init

* init and fix testing

* fix integration test

* updates test

* clean up docs

* lint notebook

* fix tests with types

* linting

* release notes

* update wording

* update impl for natural language and datetimes, remove old tests

* fix tests

* fix doctest

* release notes

* minor cleanup

* update release notes

* remove impute all

* linting
@angela97lin angela97lin mentioned this pull request Jan 25, 2022
Merged
angela97lin added a commit that referenced this pull request Jan 26, 2022
@angela97lin @chukarsten
Removes DateTimeNaNDataCheck and NaturalLanguageNaNDataCheck (#…
c46ed0c 
…3260)

* Update `validate()` API (#3142)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* fix doctests

* fix ts splitting test and impl

* fix more ts splitting tests

* fix doctest for ts data check

* release notes

* update target leakage data check docstring for consistency

* doctest linting and cleanup

* linting

* fix merging main issues

* Update `validate()` API to use `DataCheckActionOption` instead of `DataCheckAction` (#3152)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* add more tests

* Rename `HighlyNullDataCheck` to `NullDataCheck` and update data check to return impute action for non-highly null columns. (#3197)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* init rename

* retrigger

* revert release nots

* add new logic for detecting null cols

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix test and iteration for null data check

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* oops fix merge

* add more tests

* fix null data check tests

* update to use per column strategy and fix tests

* fix tests for data checks

* fix tests and doctests

* release notes

* fix release notes

* oops update action code

* oops fix test

* update wording of messages

* move logic out of dict

* update mode to most_frequent for impute strategies

* oops fix linting and doctests

* Flatten data check action ``validate`` API (#3244)

* init

* more cleanup

* begin to clean up tests

* fix more tests

* updating naming to action_options and fixing more tests

* fix no variance

* fix another test

* fixing automl tests

* oops actually fix automl tests

* fix the other automl tests

* fixing notebook

* fix data check tests

* fix doctests and docs

* integration tests and cleanup

* cleanup based on comments

* linting

* clean up notebook and tests

* Update `make_pipeline_from_actions` to handle null column imputation (#3237)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* init rename

* retrigger

* revert release nots

* add new logic for detecting null cols

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix test and iteration for null data check

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* oops fix merge

* add more tests

* fix null data check tests

* update to use per column strategy and fix tests

* fix tests for data checks

* fix tests and doctests

* release notes

* fix release notes

* init

* init and fix testing

* fix integration test

* updates test

* clean up docs

* lint notebook

* fix tests with types

* linting

* release notes

* update wording

* update impl for natural language and datetimes, remove old tests

* fix tests

* fix doctest

* release notes

* minor cleanup

* init

* release notes

* oops delete file

* oops adding line removal

Co-authored-by: chukarsten <64713315+chukarsten@users.noreply.github.com>
angela97lin added a commit that referenced this pull request Jan 26, 2022
@angela97lin
Add make_pipeline_from_data_check_output() method (#3277)
a9a1be9 
* Update `validate()` API (#3142)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* fix doctests

* fix ts splitting test and impl

* fix more ts splitting tests

* fix doctest for ts data check

* release notes

* update target leakage data check docstring for consistency

* doctest linting and cleanup

* linting

* fix merging main issues

* Update `validate()` API to use `DataCheckActionOption` instead of `DataCheckAction` (#3152)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* add more tests

* Rename `HighlyNullDataCheck` to `NullDataCheck` and update data check to return impute action for non-highly null columns. (#3197)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* init rename

* retrigger

* revert release nots

* add new logic for detecting null cols

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix test and iteration for null data check

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* oops fix merge

* add more tests

* fix null data check tests

* update to use per column strategy and fix tests

* fix tests for data checks

* fix tests and doctests

* release notes

* fix release notes

* oops update action code

* oops fix test

* update wording of messages

* move logic out of dict

* update mode to most_frequent for impute strategies

* oops fix linting and doctests

* Flatten data check action ``validate`` API (#3244)

* init

* more cleanup

* begin to clean up tests

* fix more tests

* updating naming to action_options and fixing more tests

* fix no variance

* fix another test

* fixing automl tests

* oops actually fix automl tests

* fix the other automl tests

* fixing notebook

* fix data check tests

* fix doctests and docs

* integration tests and cleanup

* cleanup based on comments

* linting

* clean up notebook and tests

* Update `make_pipeline_from_actions` to handle null column imputation (#3237)

* init

* init

* start updating tests

* add validation code for option

* remove data check updates for now

* start to clean up tests and add validate_parameter tests

* revert highly null dc

* add in more valueerror test checking

* fix test and logic for column parameters

* init

* update some tests to new API

* fix more tests

* fix doc and sparsity test

* fix integration tests

* fix doctests

* init, update no variance

* update highly null dc

* fix id column dc

* update target leakage dc

* update sparsity dc

* outliers and uniqueness dc

* update target dis dc

* update invalid target data check

* fix doctests

* fix ts splitting test and impl

* fix dc validate and tests

* fix more ts splitting tests

* fix data check actions notebook

* update to remove columns to drop

* freeze

* remove rows to drop

* update dc tests

* update doctests

* fix integration tests

* revert requirements

* update parameters to set to empty dict

* fix tests

* fix doctests

* retrigger

* revert parameter for data check option

* fix data check option test

* fix more tests from updating default parameters

* release notes

* use empty instead of none

* release notes

* cleanup unnecessary code

* move logic to data check option class and rename

* init rename

* retrigger

* revert release nots

* add new logic for detecting null cols

* update integration tests

* add tests and some cleanup

* fix tests

* fix pipeline util test

* remove unnecessary conditional

* fix test and iteration for null data check

* fix naming, need to fix tests

* fix tests

* fix doctest

* add new enums

* add logic for enums

* update files to use enum

* add in testing for invalid enum

* fix doctest by updating to_dict impl

* linting

* try with different base

* oops revert yaml

* fix tests

* remove outdated code

* oops fix merge

* add more tests

* fix null data check tests

* update to use per column strategy and fix tests

* fix tests for data checks

* fix tests and doctests

* release notes

* fix release notes

* init

* init and fix testing

* fix integration test

* updates test

* clean up docs

* lint notebook

* fix tests with types

* linting

* release notes

* update wording

* update impl for natural language and datetimes, remove old tests

* fix tests

* fix doctest

* release notes

* minor cleanup

* init

* lint and release notes

* move release note

* clean up notebook
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bchen1116 bchen1116 Awaiting requested review from bchen1116

@jeremyliweishih jeremyliweishih Awaiting requested review from jeremyliweishih

@chukarsten chukarsten Awaiting requested review from chukarsten

@dsherry dsherry Awaiting requested review from dsherry

3 more reviewers

@eccabay eccabay eccabay left review comments

@ParthivNaresh ParthivNaresh ParthivNaresh approved these changes

@freddyaboulton freddyaboulton freddyaboulton approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@angela97lin angela97lin

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@angela97lin @ParthivNaresh @freddyaboulton @eccabay