Prediction Explanations: Force Plots

[chukarsten]

Addresses #1970

Single row explanations force plots:

[codecov]

Codecov Report

Merging #2157 (d9c88c1) into main (d9fd7b1) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #2157     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        281     283      +2     
  Lines      25057   25233    +176     
=======================================
+ Hits       24957   25133    +176     
  Misses       100     100             

Impacted Files	Coverage Δ
...prediction_explanations/_report_creator_factory.py	100.0% <ø> (ø)
...nderstanding/prediction_explanations/explainers.py	100.0% <ø> (ø)
...ediction_explanations_tests/test_user_interface.py	100.0% <ø> (ø)
evalml/model_understanding/force_plots.py	100.0% <100.0%> (ø)
...derstanding/prediction_explanations/_algorithms.py	99.0% <100.0%> (+0.1%)	⬆️
...tanding/prediction_explanations/_user_interface.py	100.0% <100.0%> (ø)
...s/prediction_explanations_tests/test_algorithms.py	98.3% <100.0%> (+0.1%)	⬆️
.../prediction_explanations_tests/test_force_plots.py	100.0% <100.0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d9fd7b1...d9c88c1. Read the comment docs.

[chukarsten]

The intent behind this is to calculate the expected value.

[chukarsten]

In the similar style as drill down.

[freddyaboulton]

@chukarsten Thank you for making the changes! I think one clean-up I'd like to do is remove the _include_expected_value parameter since it's not used by the forceplot code and it doesn't seem to change the output of the explainers.

I left a comment about adding some tests for pipelines that use OHE/Text Featurizers so that we check the features are aggregated. Let me know what you think about this change.

I also think there may be a bug with the jupyter display? The following code raises TypeError: Object of type Timestamp is not JSON serializable:

from evalml.demos import load_fraud
from evalml.pipelines import BinaryClassificationPipeline
from evalml.model_understanding import explain_predictions
from evalml.model_understanding.force_plots import graph_force_plot
import pytest


X, y = load_fraud(1000)

pipeline = BinaryClassificationPipeline(["Text Featurization Component", "DateTime Featurization Component",
                                         "One Hot Encoder", "Random Forest Classifier"])
pipeline.fit(X, y)

results = graph_force_plot(pipeline, rows_to_explain=[1],
                           training_data=X, y=y)

for result in results:
    for cls in result:
        print("Class:", cls)
            display(result[cls]["plot"])

I think the issue is that the input features are timestamps:

I say we don't block merge on that bug though cause it's coming from the display call. My hypothesis is that it might work if users use their own code to display the data from force_plot. I think we should file a spike issue to investigate further. Let me know what you think.

[dsherry]

Do we have test coverage for this logic?

[dsherry]

@chukarsten looking good to me :) left a couple questions

[bchen1116]

Looking good! Left a few questions and clean-up comments.

[bchen1116]

nitpick: empty cell

[bchen1116]

How long does this take? Can we limit it to only doing the force plots for a few columns rather than all? Our docs take a while to build already

[bchen1116]

Nice! I like this

[bchen1116]

What's the difference between this and initjs.assert_called()?

[bchen1116]

why do we assert len(class_labels)==1 here? Does this imply that there's only 1 class we're doing this for on regression pipelines?