What is SHAP?
SHAP (SHapley Additive exPlanations) is a framework for explaining a model prediction with additive feature attributions derived from Shapley values for a specified coalition value function.
Quick Facts
| Full Name | SHapley Additive exPlanations |
|---|---|
| Specification | Official Specification |
How It Works
Define the coalition game and output units
Choose the exact model output, explained instance, feature players, background distribution, and coalition value. A missing feature may be marginalized independently, conditioned on observed features, or handled by a model-specific path rule; these choices answer different questions when features depend on one another. The original SHAP paper formalizes additive feature attribution and its properties.
Use an explainer whose assumptions fit
Exact enumeration costs exponentially many coalitions, so implementations use model-agnostic sampling or structure-specific algorithms. TreeSHAP can exploit tree ensembles; KernelSHAP fits a specially weighted linear model; DeepSHAP and GradientSHAP introduce neural-network approximations. The SHAP documentation also shows that explaining raw scores, probabilities, or log-odds yields different additive stories.
Validate dependence, convergence, and aggregation
Report the background sample, masker, feature groups, model and explainer versions, output units, approximation budget, and random seed. Check additive reconstruction, repeat approximate runs, vary credible backgrounds, and test whether top-ranked features affect outputs under domain-valid perturbations. Aggregated absolute SHAP values summarize model attribution over the sampled population; they do not provide effect direction, causal importance, fairness, or stability by themselves.
Key Characteristics
- Explains one model output with additive feature or feature-group contributions
- Averages marginal contributions across possible feature coalitions
- Reconstructs the explained output from a baseline plus attributions
- Supports exact, sampled, and model-structure-specific explainers
- Depends on the coalition value, masker, background data, and output space
- Provides model attribution rather than automatic real-world causal effects
Common Use Cases
- Explaining why one tabular prediction differs from a background expectation
- Comparing feature-attribution distributions across cohorts or model versions
- Inspecting tree-ensemble predictions with a structure-aware explainer
- Grouping tokens, pixels, or correlated fields into meaningful players
- Auditing explanation sensitivity to background data and dependence assumptions
Example
Loading code...Frequently Asked Questions
What is the difference between SHAP and a Shapley value?
A Shapley value is a cooperative-game solution defined for players and a value function. SHAP applies that idea to additive model explanations and includes algorithms for constructing or approximating a feature coalition game. A SHAP result therefore depends on how features, missingness, background data, and model output are defined.
Why do SHAP values sum to a model output difference?
The efficiency or local-accuracy property allocates the coalition game's total gain among its players. For model explanations, the attributions sum from the explainer's baseline value to the selected output for the instance. The statement applies in the configured output space, such as raw score or log-odds, not automatically probability.
Why does the SHAP background dataset matter?
The background defines the reference expectation and often supplies values for masked features. Changing it changes the comparison population and can change every attribution. Use a documented, representative, authorized sample for the decision being explained, test sensitivity to alternatives, and avoid leaking private records.
How do correlated features affect SHAP explanations?
Correlated features make the meaning of an absent feature ambiguous. Independent masking can create unrealistic combinations, while conditional methods may distribute credit through associations the model did not directly use. Group related features, state the dependence convention, compare plausible maskers, and avoid presenting one ranking as uniquely correct.
Are SHAP values causal effects?
Not by default. SHAP allocates a configured model prediction under a coalition value function. Terms such as interventional describe a missing-feature distribution or operation inside the explainer, not automatic identification of a real-world intervention effect. Causal claims require a causal graph, assumptions, estimand, and suitable data or experiment.