Skip to content

Refactoring deep learning models to work with any explainer#261

Merged
amit-sharma merged 18 commits intomasterfrom
amshar/method-compat-with-dl-models
Oct 6, 2022
Merged

Refactoring deep learning models to work with any explainer#261
amit-sharma merged 18 commits intomasterfrom
amshar/method-compat-with-dl-models

Conversation

@amit-sharma
Copy link
Copy Markdown
Collaborator

@amit-sharma amit-sharma commented Jan 16, 2022

[READY TO MERGE]
Refactors deep learning model and explainers so that they are compatible with the sklearn explainers. sklearn explainers do not need access to gradient, so they should also work for deep learning models.

  • DL and sklearn models need to have the same output
  • DL methods employ user-provided encoding of features
  • model needs to handle the conversion from raw to one hot encoded data
  • Remove any native calls to DL model's methods
  • Updated tests, added torch as a test dependency

Possible fix to #246

@amit-sharma amit-sharma requested a review from gaugup as a code owner January 16, 2022 04:54
@amit-sharma amit-sharma removed the request for review from gaugup January 16, 2022 04:54
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 16, 2022

Unit Test Results

261 tests  +14   256 ✔️ +14   19m 21s ⏱️ + 8m 27s
    1 suites ±  0       5 💤 ±  0 
    1 files   ±  0       0 ±  0 

Results for commit d0127b4. ± Comparison against base commit 2e9d093.

This pull request removes 29 and adds 43 tests. Note that renamed tests count towards both.
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_features_to_vary[1-2-features_to_vary0-kdtree]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_features_to_vary[1-2-features_to_vary1-random]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_invalid_query_instance[features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_maxiter[0-7-kdtree-0]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_maxiter[0-7-random-0]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_no_cfs[7-3]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range[1-2-all-permitted_range0-kdtree]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range[1-2-all-permitted_range1-random]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range_categorical[1-2-all-permitted_range0-kdtree]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range_categorical[1-2-all-permitted_range1-random]
…
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_features_to_vary[sklearn-1-2-features_to_vary0-kdtree]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_features_to_vary[sklearn-1-2-features_to_vary1-random]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_invalid_query_instance[sklearn-features_to_vary0-permitted_range0-inverse_mad]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_maxiter[sklearn-0-7-kdtree-0]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_maxiter[sklearn-0-7-random-0]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_no_cfs[sklearn-7-3]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range[sklearn-1-2-all-permitted_range0-kdtree]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range[sklearn-1-2-all-permitted_range1-random]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range_categorical[sklearn-1-2-all-permitted_range0-kdtree]
tests.test_dice_interface.test_dice_genetic.TestDiceGeneticBinaryClassificationMethods ‑ test_permitted_range_categorical[sklearn-1-2-all-permitted_range1-random]
…

♻️ This comment has been updated with latest results.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jul 10, 2022

Codecov Report

❌ Patch coverage is 87.50000% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.92%. Comparing base (2e9d093) to head (d0127b4).

Files with missing lines Patch % Lines
dice_ml/explainer_interfaces/dice_tensorflow1.py 0.00% 11 Missing ⚠️
dice_ml/data_interfaces/private_data_interface.py 82.75% 5 Missing ⚠️
dice_ml/dice.py 86.36% 3 Missing ⚠️
dice_ml/model_interfaces/pytorch_model.py 89.65% 3 Missing ⚠️
dice_ml/explainer_interfaces/dice_genetic.py 81.81% 2 Missing ⚠️
dice_ml/data_interfaces/base_data_interface.py 96.42% 1 Missing ⚠️
dice_ml/data_interfaces/public_data_interface.py 87.50% 1 Missing ⚠️
dice_ml/explainer_interfaces/explainer_base.py 96.42% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #261      +/-   ##
==========================================
+ Coverage   71.47%   71.92%   +0.44%     
==========================================
  Files          25       26       +1     
  Lines        3464     3537      +73     
==========================================
+ Hits         2476     2544      +68     
- Misses        988      993       +5     
Flag Coverage Δ
unittests 71.92% <87.50%> (+0.44%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Amit Sharma <amit_sharma@live.com>
@amit-sharma amit-sharma requested a review from gaugup July 11, 2022 04:08
@amit-sharma amit-sharma changed the title [WIP] Refactoring deep learning models to work with any explainer Refactoring deep learning models to work with any explainer Jul 11, 2022
Signed-off-by: Amit Sharma <amit_sharma@live.com>
@amit-sharma amit-sharma merged commit 1651751 into master Oct 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants