How does multicollinearity affect DeepExplainer?

[jaxondk]

The original SHAP paper states that "DeepLIFT approximates SHAP values assuming that the input features are independent of one another and the deep model is linear." However, I am working with tabular data that (like almost all tabular data) has some features that are correlated with each other. I am restricted to using deep learning models and therefore cannot simply use a TreeExplainer with feature_perturbation=“tree_path_dependent”.

My question is two-fold:

1. How does multicollinearity affect DeepSHAP explanations? IE can you reasonably trust SHAP values even in a multicollinear environment?
2. Related, are there known workarounds? For example, if I know certain features are correlated, can I interpret the SHAP values accordingly? Or can I, for example, pair them when calculating shap values somehow, as some do when doing permutation importance?

The original SHAP paper also states that "when using these methods [model-specific methods, including DeepExplainer], feature independence and model linearity are two optional assumptions simplifying the computation of the expected value..." How can I exercise the option to not assume feature independence when using DeepExplainer, and what are the consequences when it comes to time complexity?

Many thanks!

References

• I have briefly skimmed this paper, which critiques SHAP and I believe is related to my question. It mentions that "parts of the package SHAP from Lundberg and Lee (2017) are unaffected by this... since they 'approximate' the observational expectations by an expression that would've been the right one in the first place." But I wasn't entirely sure if this was totally related, nor which parts of SHAP are affected.
• This comment on a thread at fast.ai gives a good overview of this question. There was no real conclusion, though it was suggested in a related thread that you don't really need to satisfy the condition of independence of features. I want to know if that is fairly accurate, or if shap values can't really be trusted in a multicollinear environment.

[jaxondk]

I'm copying my post from here as it addresses this topic somewhat. I would love to have some of this confirmed @slundberg if you can. I pulled from a few various places for this information, and it may be useful for others to have it all in one place and to have an authoritative stamp of approval if it's correct. Also, I'd love to know how to do observational conditional expectations using DeepSHAP or some other way so that I can be "true to the data" when required.

[Beginning copied post]

Here are my notes on just a couple of these papers as well as some very useful threads/comments in various locations. Please realize that I skimmed these papers fairly quickly, and I also don't have a PhD and am fairly removed from academia, so I don't read massive amounts of papers on a regular basis. All that being said, I could totally be misinterpreting things and would love for someone to correct me on anything here :) I feel fairly confident about the first one, and honestly barely looked at the second but included a small blurb about it anyway. I could be oversimplifying things in my conclusion section, so please provide some feedback so I don't mislead the community or myself 😆

True to the Model or True to the Data

Discusses observational vs interventional conditional expectations when doing feature importance, and how neither is preferred generally but rather it's application-specific, and depends on if you want to be "true to the model" or "true to the data". Things like permutation importance and the SHAP approximations in DeepSHAP are interventional (seems Lundberg, author of shap, agrees), or "true to the model".

The paper states that if you have independent features, importance values are the same for observational and interventional. Otherwise, "correlation splits the Beta as credit between correlated variables and higher levels of correlation leads to slower convergence of the observational Shapley value estimates." However, I don't know if this means they split it evenly between the correlated features. If it weren't split evenly, then you may have one correlated feature look more important than the other features it's correlated with disproportionately. However, this splitting only occurs if doing observational. This is because in interventional, it only shows you what the model is actually using. If it only uses one of the correlated features, it won't split the credit between the other correlated features.

Conclusion: Essentially, "true to the model" = interventional, "true to the data" = observational. "Being true to the model is the best choice for most applications of explainable AI, where the goal is to explain the model itself." However, if you are "focused on scientific discovery", you will likely want to be true to the data.
The authors note that in an ideal world, you can reparameterize your model to get at the underlying independent factors. Then you can use interventional techniques without getting off the data manifold.

Understanding Global Feature Contributions

I ended up not spending too much time on this one. I was excited about it, as it seemed to be a method for handling correlated features when doing global feature importance, but then read this:

In practice we sample from the marginal distribution, which corresponds to an assumption of
feature independence

I had thought this paper was about dealing with feature dependence. I stopped reading at this point, though to be honest I'm sure I'm missing the point on this one.

Scouring various threads

• "SHAP aligns with casual interventional perturbations" (Lundberg). Honestly, that whole thread was very useful
• In the presence of correlated features, you cannot be both true to the data and true to the model. Either you must provide some data as inputs that are off-manifold (if you use interventional), or you must allow credit to bleed between correlated features (if you use observational). This is also the main gist of the "true to model / true to data" paper. (Lundberg)
• This comment discusses some of the differences between observational and interventional, and also provided links to some of these other threads.

The Takeaway

This was really helpful for me. I would suspect that most practitioners here are more concerned about being true to the model. If that's the case, it seems that things like permutation importance or SHAP values will work for you, even in the presence of correlated features. You will just see what the model used, regardless of multicollinearity.

For those more interested in scientific discovery, it looks like observational shapley values are what you want. Currently, I'm unaware of a package for calculating these for neural nets. Maybe SHAP has this functionality optionally that I'm unaware of. I would be very interested in learning about a package to compute these, as I am involved in both simply explaining a model and scientific discovery.

[berlinshawn063]

Multicollinearity is when independent variables in a regression model are correlated. I explore its problems, testing your model like check amazon gift card balance without redeeming for it, and solutions.

[jaxondk]

Reading Explaining Individual Predictions when Features are Dependent: More Accurate Approximations to Shapley Values, however, gives me pause. It seems that even when doing the interventional approach, the shap values can be incorrect in the presence of feature dependence. I notice that this paper was published before your "True to the Model/Data" one, however. If we are concerned with being true to the model, do the objections brought up in this earlier paper not apply? SHAP values will give an accurate picture of what the model is actually using regardless of feature dependence, correct?

[SonLoc]

Hello @jaxondk ,
Did you ever solve your question? I have stumbled upon the same issue when working on my thesis and was looking for an answer to the exact same question. I also found this paper; Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability Interesting.

[jaxondk]

Unfortunately I don't have any updates for you beyond what I wrote above :/ Some priorities changed at work and I haven't been doing as much interpretability the last month or so. However, I'm relying mostly on the explanation in the true to the model vs. true to the data paper that SHAP values will actually be true to the model. Please let us know if you find out more, and good luck on your thesis!