Repository navigation
Time Series: Explain Predictions - #1818
Conversation
9c85412 to
db9d8c2
Compare
Codecov Report
@@ Coverage Diff @@
## main #1818 +/- ##
=======================================
+ Coverage 99.9% 99.9% +0.1%
=======================================
Files 255 255
Lines 20571 20608 +37
=======================================
+ Hits 20549 20586 +37
Misses 22 22
Continue to review full report at Codecov.
|
| # More than 100 datapoints can negatively impact runtime according to SHAP | ||
| # https://github.com/slundberg/shap/blob/master/shap/explainers/kernel.py#L114 | ||
| sampled_training_data_features = pipeline.compute_estimator_features(shap.sample(training_data, 100)).to_dataframe() | ||
| sampled_training_data_features = shap.sample(training_data, 100) |
There was a problem hiding this comment.
Doing this to avoid having to compute the features twice.
freddyaboulton
left a comment
There was a problem hiding this comment.
@chukarsten Looks good to me! It was a pleasure getting this done together. I just have some minor clean-up style comments.
| "target_value": _make_json_serializable(y_true.iloc[index]), | ||
| "error_name": self.error_name, | ||
| "error_value": _make_json_serializable(scores.iloc[index]), | ||
| "error_value": _make_json_serializable(scores[index]), |
There was a problem hiding this comment.
This is because the error metrics are now stored in a numpy array.
|
|
||
| # Check that the computed features to be explained aren't NaN. | ||
| for exp_idx in range(len(exp["explanations"])): | ||
| assert not np.isnan(np.array(exp["explanations"][exp_idx]["explanations"][0]["feature_values"])).any() |
There was a problem hiding this comment.
We're not checking the output against a specified value. The other tests check that the logic for selecting the rows that are explained and displaying the right features/shap values is correct so I think running real data through and verifying it completes without showing NaNs is sufficient.
6a5f70c to
b556cee
Compare
591b795 to
cb0e9cc
Compare
Addresses: #1504
Oh my goodness. You've been waiting for it, so here it is.