953 implement table ui for shap #974

freddyaboulton · 2020-07-24T13:52:03Z

Pull Request Description

Closes #953 .

Demo of what is displayed to user

To see more examples, see https://github.com/FeatureLabs/shap-reports/pull/1 (but those results include the SHAP value).

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

…ependencies.

codecov · 2020-07-24T13:53:20Z

Codecov Report

Merging #974 into main will increase coverage by 0.20%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #974      +/-   ##
==========================================
+ Coverage   99.67%   99.87%   +0.20%     
==========================================
  Files         174      178       +4     
  Lines        9043     9163     +120     
==========================================
+ Hits         9014     9152     +138     
+ Misses         29       11      -18

Impacted Files	Coverage Δ
...l/pipelines/prediction_explanations/_algorithms.py	`100.00% <100.00%> (ø)`
...l/pipelines/prediction_explanations/_explainers.py	`100.00% <100.00%> (ø)`
...pelines/prediction_explanations/_user_interface.py	`100.00% <100.00%> (ø)`
...peline_tests/explanations_tests/test_explainers.py	`100.00% <100.00%> (ø)`
...ne_tests/explanations_tests/test_user_interface.py	`100.00% <100.00%> (ø)`
evalml/tests/utils_tests/test_cli_utils.py	`100.00% <100.00%> (ø)`
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 967073c...ff81520. Read the comment docs.

…elines/explanations.

freddyaboulton · 2020-07-24T18:44:56Z

core-requirements.txt

@@ -11,3 +11,4 @@ click>=7.0.0
 psutil>=5.6.3
 requirements-parser>=0.2.0
 shap>=0.35.0
+texttable>=1.6.2


The wheel is 10kb and it's implemented in base python so I don't think it will cause any dependency issues down the line.

freddyaboulton · 2020-07-24T18:46:00Z

evalml/pipelines/prediction_explanations/_explainers.py

+    return _make_single_prediction_table(shap_values, normalized_shap_values, top_k, include_shap_values)
+
+
+def _explain_prediction(pipeline, features, training_data=None):


We would make this function public once we go live. I don't think we'd make _explain_with_shap_values public but if someone ever needed to access the SHAP values, we can point them to it.

@freddyaboulton I have a suggestion on the organization:

Either delete this method, or name it print_prediction_explanation. Personally I think its ok to delete since users can call print whenever they want

Rename _explain_with_shap_values to _explain_prediction

Yeah, deleting the one which calls print will make the testing easier too

+1 to this!

dsherry · 2020-07-24T21:34:04Z

evalml/pipelines/prediction_explanations/_algorithms.py

@@ -52,7 +52,9 @@ def _compute_shap_values(pipeline, features, training_data=None):

    # This is to make sure all dtypes are numeric - SHAP algorithms will complain otherwise.
    # Sklearn components do this under-the-hood so we're not changing the data the model was trained on.
-    pipeline_features = check_array(pipeline_features.values)
+    # Catboost can naturally handle string-enconded categorical features so we don't need to convert to numeric.


Ah, makes sense.

Typo: string-enconded --> string-encoded

dsherry

Great stuff! 👏 The table looks really nice! I like that the logic to compute it is pretty easy to follow. I'm excited to try this out.

Nothing blocking per se, but two thoughts which would be good to resolve:

I do think it would be a good idea to resolve the discussion about method names in _explainers.py. I suggested deleting the method which just calls print and renaming the other one to _explain_prediction
I think your test coverage of make_table is solid, but it would be great to add coverage of what's currently called _explain_with_shap_values, assuming we delete _explain_predictions. Could be as simple as mocking _compute_shap_values and asserting the returned string matches on one example.

There's a lot of standalone methods at this point for prediction explanation. Makes me wonder if a class would help with encapsulation. But nothing jumped out at me... fundamentally, having everything wrapped up in explain_prediction means users don't need to worry about the rest.

You're gonna add docs and example usage in a separate PR, right?

dsherry · 2020-07-24T21:41:57Z

evalml/pipelines/prediction_explanations/_explainers.py

+    return _make_single_prediction_table(shap_values, normalized_shap_values, top_k, include_shap_values)
+
+
+def _explain_prediction(pipeline, features, training_data=None):


@freddyaboulton I have a suggestion on the organization:

Either delete this method, or name it print_prediction_explanation. Personally I think its ok to delete since users can call print whenever they want

Rename _explain_with_shap_values to _explain_prediction

dsherry · 2020-07-24T21:42:28Z

evalml/pipelines/prediction_explanations/_explainers.py

+        None: displays a table to std out
+    """
+    if not (isinstance(features, pd.DataFrame) and features.shape[0] == 1):
+        raise ValueError("features must be stored in a dataframe of one row.")


Perhaps if we delete this method, this validation can be moved into the other method

I'm on-board with the re-org you suggested. The only reason I did it this way was to not directly expose the ability to display SHAP values as a column but I don't think that's necessary anymore.

dsherry · 2020-07-24T21:43:45Z

evalml/pipelines/prediction_explanations/_explainers.py

+        features (pd.DataFrame): Dataframe of features - needs to correspond to data the pipeline was fit on.
+        top_k (int): How many of the highest/lowest features to include in the table.
+        training_data (pd.DataFrame): Training data the pipeline was fit on.
+            For non-tree estimators, we need a sample of training data for the KernelSHAP algorithm.


Nice. Suggest "Training data the pipeline was fit on. This is required for non-tree estimators because we need a sample of training data for the KernelSHAP algorithm."

dsherry · 2020-07-24T21:50:16Z

evalml/pipelines/prediction_explanations/_explainers.py

+
+    Arguments:
+        pipeline (PipelineBase): Fitted pipeline whose predictions we want to explain with SHAP.
+        features (pd.DataFrame): Dataframe of features - needs to correspond to data the pipeline was fit on.


This is one row of input data, right? Why not expect pd.Series? Perhaps we can say "input features" to make it clear we're talking about the raw input.

>>> data = pd.read_csv("/Users/dylan.sherry/Downloads/dataset_61_iris.csv") >>> row0 = data.loc[0] >>> print(row0) >>> print() >>> print(type(row0)) >>> print() >>> print(row0.index) >>> print() >>> print(row0.to_numpy())

outputs

sepallength 5.1 sepalwidth 3.5 petallength 1.4 petalwidth 0.2 class Iris-setosa Name: 0, dtype: object <class 'pandas.core.series.Series'> Index(['sepallength', 'sepalwidth', 'petallength', 'petalwidth', 'class'], dtype='object') [5.1 3.5 1.4 0.2 'Iris-setosa']

The problem is the call to pipeline._transform. It can't handle pd.Series objects so we would have to convert to a dataframe internally. The problem with doing the conversion internally is that the dtype information is lost when you do df.iloc[index] (see the example below). This would cause some of the components to freak out (like OneHotEncoder) because there are more object dtype columns than the data it was fit on.

We would have to require the user to always pass in the dtypes and/or the training data? This would only impact explaining tree based models which are the only estimators for which SHAP doesn't require training data. It's doable but I'm curious what your thoughts are.

Got it @freddyaboulton . So then pandas dataframe is required in this method?

dsherry · 2020-07-24T21:51:45Z

evalml/pipelines/prediction_explanations/_user_interface.py

+    rows = []
+    for value, feature_name in features_to_display:
+        symbol = "+" if value >= 0 else "-"
+        display_text = symbol * min(int(abs(value) // 0.2) + 1, 5)


Nice!! Breaking out the integer division operator, love it 😁

dsherry · 2020-07-24T21:54:03Z

evalml/tests/pipeline_tests/explanations_tests/test_explainers.py

+test_cases = [5, [1], np.ones((1, 15)), pd.DataFrame({"a": [1, 2, 3], "b": [1, 2, 3]}).iloc[0]]
+
+
+@pytest.mark.parametrize("test_case", test_cases)


Nit-pick: one's singular and one's plural

dsherry · 2020-07-24T21:54:52Z

evalml/pipelines/prediction_explanations/_explainers.py

+    return _make_single_prediction_table(shap_values, normalized_shap_values, top_k, include_shap_values)
+
+
+def _explain_prediction(pipeline, features, training_data=None):


Yeah, deleting the one which calls print will make the testing easier too

dsherry · 2020-07-24T21:56:22Z

evalml/tests/pipeline_tests/explanations_tests/test_user_interface.py

+                         'c + 0.000', 'd -- -2.560', 'e -- -2.800', 'f -- -2.900', '', '', 'Class 2', '',
+                         'Feature Name Contribution to Prediction SHAP Value',
+                         '======================================================', 'a + 0.680', 'c + 0.000',
+                         'b + 0.000', 'd -- -1.840', 'e -- -2.040', 'f -- -2.680', '']


Idk if this makes things easier but you could do

multiclass_table_shap = """Class 0 Feature Name Contribution to Prediction SHAP Value ====================================================== ... """.split(\n)

Much more legible!

dsherry · 2020-07-24T21:56:41Z

evalml/tests/pipeline_tests/explanations_tests/test_user_interface.py

+
+    # Making sure the content is the same, regardless of formatting.
+    for row_table, row_answer in zip(table.splitlines(), answer):
+        assert row_table.strip().split() == row_answer.strip().split()


This is great!

dsherry · 2020-07-24T21:57:02Z

evalml/tests/pipeline_tests/explanations_tests/test_user_interface.py

+    table = _make_table(dtypes, alignment, values, values, top_k, include_shap_values).splitlines()
+    if include_shap_values:
+        assert "SHAP Value" in table[0]
+    # Subtracting two because a header and a line under the header are included in the table.


…es to _explain_prediction.

angela97lin

This looks really good! 👍

freddyaboulton · 2020-07-27T15:20:35Z

@dsherry Thanks for the feedback! Yes, I plan on adding a tutorial and updating the docs in the next PR once this is merged. I think using classes might make more sense when we have multiple interpretation algorithms but I think the code layout here is straightforward and there isn't a lot "state" we need to keep track of yet.

I renamed _explain_with_shap_values to explain_prediction, added a proper test case for it, and followed your other suggestions.

freddyaboulton added 14 commits July 23, 2020 17:05

Implementing _compute_shap_values and _normalize_values.

0256303

Updating release notes for PR 958.

65105e4

Adding shap to core requirements

ebd2058

Skipping test cases for test_value_errors_raised that dont use core d…

51ca19f

…ependencies.

Fixing import order in _algorithms.

861ff0e

Changing how SHAP values are normalized.

c2761cd

Fixing typo in ValueError raised by _normalize_shap_values.

63d4d0a

Adding is_tree_estimator to ModelFamily

0c7ddd6

Logging warnings.

8163499

Fixing typo in example for _normalize_values_dict.

88ba8f4

Implementing table output for prediction explanations.

f88c140

Adding support for classification problems.

887b661

Adding testing for user interface.

66380ce

Adding _explain_prediction and unit tests.

bac41eb

freddyaboulton self-assigned this Jul 24, 2020

freddyaboulton added 2 commits July 24, 2020 09:54

Updating release notes for PR 974.

505b2df

Merge branch 'main' into 953-implement-table-ui-for-shap

449da44

freddyaboulton changed the base branch from main to master July 24, 2020 17:25

freddyaboulton changed the base branch from master to main July 24, 2020 17:25

freddyaboulton closed this Jul 24, 2020

Move ui code from explanations to prediction_explanations deleted pip…

c77240d

…elines/explanations.

freddyaboulton reopened this Jul 24, 2020

freddyaboulton commented Jul 24, 2020

View reviewed changes

freddyaboulton marked this pull request as ready for review July 24, 2020 19:00

freddyaboulton requested review from dsherry, angela97lin and jeremyliweishih July 24, 2020 19:00

dsherry reviewed Jul 24, 2020

View reviewed changes

dsherry approved these changes Jul 24, 2020

View reviewed changes

freddyaboulton added 3 commits July 24, 2020 18:32

Merging latest changes from main.

c9a3d8b

Fixing test_get_core_requirements.

897dec7

Deleting old _explain_prediction and renaming _explain_with_shap_valu…

ac19087

…es to _explain_prediction.

angela97lin approved these changes Jul 27, 2020

View reviewed changes

Merge branch 'main' into 953-implement-table-ui-for-shap

ff81520

freddyaboulton merged commit b71cb22 into main Jul 27, 2020

freddyaboulton deleted the 953-implement-table-ui-for-shap branch July 28, 2020 15:11

angela97lin mentioned this pull request Jul 31, 2020

Release v0.12.0 #1007

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

953 implement table ui for shap #974

953 implement table ui for shap #974

freddyaboulton commented Jul 24, 2020 •

edited

Loading

codecov bot commented Jul 24, 2020 •

edited

Loading

freddyaboulton Jul 24, 2020

freddyaboulton Jul 24, 2020 •

edited

Loading

dsherry Jul 24, 2020

dsherry Jul 24, 2020

angela97lin Jul 27, 2020

freddyaboulton Jul 27, 2020

dsherry Jul 24, 2020

dsherry left a comment

dsherry Jul 24, 2020

dsherry Jul 24, 2020

freddyaboulton Jul 27, 2020

dsherry Jul 24, 2020

dsherry Jul 24, 2020

freddyaboulton Jul 27, 2020 •

edited

Loading

dsherry Jul 28, 2020

dsherry Jul 24, 2020

dsherry Jul 24, 2020

dsherry Jul 24, 2020

dsherry Jul 24, 2020

freddyaboulton Jul 27, 2020

dsherry Jul 24, 2020

dsherry Jul 24, 2020

angela97lin left a comment

freddyaboulton commented Jul 27, 2020 •

edited

Loading

		return _make_single_prediction_table(shap_values, normalized_shap_values, top_k, include_shap_values)


		def _explain_prediction(pipeline, features, training_data=None):

		test_cases = [5, [1], np.ones((1, 15)), pd.DataFrame({"a": [1, 2, 3], "b": [1, 2, 3]}).iloc[0]]


		@pytest.mark.parametrize("test_case", test_cases)

953 implement table ui for shap #974

953 implement table ui for shap #974

Conversation

freddyaboulton commented Jul 24, 2020 • edited Loading

Pull Request Description

Demo of what is displayed to user

codecov bot commented Jul 24, 2020 • edited Loading

Codecov Report

Choose a reason for hiding this comment

freddyaboulton Jul 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dsherry left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

freddyaboulton Jul 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

angela97lin left a comment

Choose a reason for hiding this comment

freddyaboulton commented Jul 27, 2020 • edited Loading

freddyaboulton commented Jul 24, 2020 •

edited

Loading

codecov bot commented Jul 24, 2020 •

edited

Loading

freddyaboulton Jul 24, 2020 •

edited

Loading

freddyaboulton Jul 27, 2020 •

edited

Loading

freddyaboulton commented Jul 27, 2020 •

edited

Loading