SHAP

Definition

$${\Phi }_i(f,x)=\sum _{S\subseteq F\setminus \{i\}}\frac{|S|!(|F|-|S|-1)!}{|F|!}[v(x,S\cup \{i\})-v(x,S)]$$

(Lundberg and Lee 2017). Efficiency: ${\sum }_i{\Phi }_i=f(x)-Ef(X)$.

Formula

• Observational: $v_{obs}(x,S)=E[f(X)\mid X_S=x_S]$.
• Interventional: $v_{int}(x,S)=E_{X_{\bar{S}}}f(x_S,X_{\bar{S}})$ (marginal, breaks dependencies, out-of-distribution points).
• GAM with centered components, interventional: ${\Phi }_i=f_i(x_i)$.

Two features: weights $\frac{1}{2},\frac{1}{2}$. Example $f=2x_1+x_2+3x_1{x}_2$, independent uniform bits, $x=(1,1)$: ${\Phi }_1=2.125$, ${\Phi }_2=1.625$.

Both value functions coincide for independent features. With interactions SHAP splits the interaction into main effects; with dependencies it spreads importance over correlated features (observational SHAP credits an unused duplicate). The lecturer’s advice: avoid SHAP unless you really know what you are doing.

Appears in

• Lecture 9.2, definition, pizza illustration
• Lecture 9.2, value functions
• Lecture 9.2, Theorem 1 (GAMs)
• Lecture 9.2, exam-style task
• Lecture 9.2, failure modes

Related concepts

• Feature Attribution
• LIME