Displaying the top_k features with largest shap value magnitudes by freddyaboulton · Pull Request #1374 · alteryx/evalml

freddyaboulton · 2020-10-30T15:38:19Z

Pull Request Description

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

codecov · 2020-10-30T16:35:08Z

Codecov Report

Merging #1374 (2afa992) into main (4bf2f9b) will decrease coverage by 0.1%.
The diff coverage is 100.0%.

@@            Coverage Diff            @@
##             main    #1374     +/-   ##
=========================================
- Coverage   100.0%   100.0%   -0.0%     
=========================================
  Files         213      213             
  Lines       13942    13940      -2     
=========================================
- Hits        13935    13933      -2     
  Misses          7        7

Impacted Files	Coverage Δ
...s/prediction_explanations_tests/test_explainers.py	`100.0% <ø> (ø)`
...tanding/prediction_explanations/_user_interface.py	`100.0% <100.0%> (ø)`
...ediction_explanations_tests/test_user_interface.py	`100.0% <100.0%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4bf2f9b...2afa992. Read the comment docs.

bchen1116

LGTM

bchen1116 · 2020-11-02T14:58:59Z

   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.7.5"
+   "version": "3.8.3"


this doesnt need to be changed

bchen1116 · 2020-11-02T15:19:49Z

+
+    # Sort the features s.t the top_k w the largest shap values are the firt
+    # top_k elements
+    tuples = sorted(tuples, key=_neg_shap_value_magnitude)


this is really cool!

angela97lin

LGTM, left a small comment about a comment typo and how we can simplify to not need to define _neg_shap_value_magnitude but nothing blocking :d

angela97lin · 2020-11-03T23:18:05Z

+    def _neg_shap_value_magnitude(shap_tuple):
+        return -abs(shap_tuple[0])
+
+    # Sort the features s.t the top_k w the largest shap values are the firt


Typo? the top_k w the largest shap values are the firt --> the top_k with the largest shap values are the first

angela97lin · 2020-11-03T23:20:33Z

-        features_to_display = reversed(tuples)
-    else:
-        features_to_display = tuples[-top_k:][::-1] + tuples[:top_k][::-1]
+    def _neg_shap_value_magnitude(shap_tuple):


Is this done strictly to sort the features w/ largest SHAP values first? If so, maybe we don't need this, and can use:

tuples = sorted(tuples, key=lambda x: x[0], reverse=True)

(Syntax not guaranteed ahaha)

I originally went with a def instead of a lambda because I thought our linting prevented lambdas but it prevents assigning to a lambda, not all lambdas.

So this is not allowed:

first_value_abs = lambda x: abs(x[0]) tuples = sorted(tuples, key=first_value, reverse=True)

But your suggestion is allowed! I guess the intention is to keep lambdas anonymous which makes sense.

TIL something hehe. I'll update to use the in-line lambda.

CLAassistant · 2020-11-04T14:42:00Z

All committers have signed the CLA.

…nitude.

dsherry · 2020-11-06T22:27:01Z

    * Enhancements
        * Added ability to freeze hyperparameters for AutoMLSearch :pr:`1284`
        * Added the index id to the ``explain_predictions_best_worst`` output to help users identify which rows in their data are included :pr:`1365`
+        * The top_k features displayed in ``explain_predictions_*`` functions are now determined by the magnitude of shap values as opposed to the ``top_k`` largest and smallest shap values. :pr:`1374`


dsherry · 2020-11-06T22:27:05Z

+.. warning::
+
+    **Breaking Changes**
+        * The ``top_k`` and ``top_k_features`` parameters in ``explain_predictions_*`` functions now return ``k`` features as opposed to ``2 * k`` features :pr:`1374`


freddyaboulton marked this pull request as ready for review October 30, 2020 17:04

freddyaboulton requested review from angela97lin, bchen1116, christopherbunn, dsherry, eccabay and jeremyliweishih October 30, 2020 19:04

freddyaboulton force-pushed the 1360-sort-by-magnitude-of-shap branch 2 times, most recently from d594e60 to 41761ba Compare October 30, 2020 20:21

freddyaboulton self-assigned this Oct 30, 2020

bchen1116 approved these changes Nov 2, 2020

View reviewed changes

angela97lin approved these changes Nov 3, 2020

View reviewed changes

freddyaboulton force-pushed the 1360-sort-by-magnitude-of-shap branch 4 times, most recently from edd16df to a15f514 Compare November 5, 2020 20:50

freddyaboulton mentioned this pull request Nov 6, 2020

Try rounding codecov down instead of up #1413

Merged

freddyaboulton force-pushed the 1360-sort-by-magnitude-of-shap branch from a15f514 to 080e232 Compare November 6, 2020 16:38

freddyaboulton added 7 commits November 6, 2020 13:28

Displaying the top_k features with largest shap value magnitudes

46dd988

Adding PR 1360 to release notes.

b020e35

Updating broken tests in test_explainers to reflect top_k rows by mag…

9254bb5

…nitude.

Updating comment.

e20008b

Removing linebreak in release notes for pr 1374.

6796e99

Sorting shap values in _make_rows via lambda function.

f3b6242

Not changing notebook python version in model understanding user guide.

2afa992

freddyaboulton force-pushed the 1360-sort-by-magnitude-of-shap branch from 080e232 to 2afa992 Compare November 6, 2020 18:29

freddyaboulton merged commit 6a80b40 into main Nov 6, 2020

dsherry reviewed Nov 6, 2020

View reviewed changes

dsherry mentioned this pull request Nov 24, 2020

Release v0.16.0 #1468

Merged

freddyaboulton deleted the 1360-sort-by-magnitude-of-shap branch June 3, 2021 14:03

Navigation Menu

Uh oh!

Displaying the top_k features with largest shap value magnitudes - #1374

Displaying the top_k features with largest shap value magnitudes#1374
freddyaboulton merged 7 commits into
mainfrom
1360-sort-by-magnitude-of-shap

freddyaboulton commented Oct 30, 2020 •

edited

Loading

Uh oh!

codecov Bot commented Oct 30, 2020 •

edited

Loading

Uh oh!

bchen1116 left a comment

Uh oh!

bchen1116 Nov 2, 2020

Uh oh!

freddyaboulton Nov 4, 2020

Uh oh!

bchen1116 Nov 2, 2020

Uh oh!

angela97lin left a comment

Uh oh!

angela97lin Nov 3, 2020

Uh oh!

angela97lin Nov 3, 2020

Uh oh!

freddyaboulton Nov 4, 2020 •

edited

Loading

Uh oh!

CLAassistant commented Nov 4, 2020 •

edited

Loading

Uh oh!

dsherry Nov 6, 2020

Uh oh!

dsherry Nov 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

freddyaboulton commented Oct 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Description

Uh oh!

codecov Bot commented Oct 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bchen1116 left a comment

Choose a reason for hiding this comment

Uh oh!

bchen1116 Nov 2, 2020

Choose a reason for hiding this comment

Uh oh!

freddyaboulton Nov 4, 2020

Choose a reason for hiding this comment

Uh oh!

bchen1116 Nov 2, 2020

Choose a reason for hiding this comment

Uh oh!

angela97lin left a comment

Choose a reason for hiding this comment

Uh oh!

angela97lin Nov 3, 2020

Choose a reason for hiding this comment

Uh oh!

angela97lin Nov 3, 2020

Choose a reason for hiding this comment

Uh oh!

freddyaboulton Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CLAassistant commented Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dsherry Nov 6, 2020

Choose a reason for hiding this comment

Uh oh!

dsherry Nov 6, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

freddyaboulton commented Oct 30, 2020 •

edited

Loading

codecov Bot commented Oct 30, 2020 •

edited

Loading

freddyaboulton Nov 4, 2020 •

edited

Loading

CLAassistant commented Nov 4, 2020 •

edited

Loading