MC Shapley + Sobol from a Single Data Collection¶

The Owen & Prieur (2017) formulation for correlated-input Shapley effects uses the cost function:

$$v(u) = \text{Cov}[f(\mathbf{X}), f(\mathbf{X}_u)]$$

where $\mathbf{X}_u$ shares the same background variables as $\mathbf{X}$ but conditions on the subset $u$. Remarkably, this same quantity equals the closed Sobol index:

$$v(u) = \mathbb{V}[\mathbb{E}(f(\mathbf{X}) \mid \mathbf{X}_u)]$$

So from the same Monte Carlo data used for Shapley effects, we can extract both first-order ($S_i$) and total-order ($T_i$) Sobol indices at zero additional cost.

$$S_i = \frac{v(\{i\})}{v(\text{full})}, \qquad T_i = 1 - \frac{v(\text{all}\\setminus\{i\})}{v(\text{full})}$$

This notebook demonstrates the MCShapley.compute() method which now returns sobol_first and sobol_total columns alongside the Shapley effects.

We use the Ishigami function with correlated inputs as the test case.

Correlated Input Distribution¶

Uniform marginals on $[-\pi, \pi]$ with $X_1$–$X_3$ correlated ($\rho = 0.8$) via a Gaussian copula.

MC Shapley + Sobol in One Call¶

MCShapley.compute() with method='exhaustive' now returns a DataFrame with sobol_first and sobol_total columns — no extra model evaluations.

Visual Comparison: Shapley vs Sobol¶

For correlated inputs, the relationship between indices becomes informative:

• $S_i$ (first-order): main effect of $X_i$ alone — ignores interactions
• Shapley: fairly distributes interaction variance among participants
• $T_i$ (total-order): effect of $X_i$ including all interactions

Why $S_i > T_i$ Can Happen Under Correlation¶

With independent inputs, Sobol indices always satisfy $S_i \leq T_i$ — the total effect includes interactions on top of the main effect. Under correlation, this inequality no longer holds.

• $S_i$ is inflated when $X_i$ is correlated with other influential variables — knowing $X_i$ also tells you about them.
• $T_i$ is deflated when $X_{-i}$ can proxy for $X_i$ through correlation — $X_i$ adds little beyond what the others already explain.

Here $X_1$ and $X_3$ are correlated ($\rho=0.8$), so:

• $S_1$ and $S_3$ are boosted (each proxies the other)
• $T_1$ and $T_3$ are reduced (knowing $\{X_2,X_3\}$ already captures much of $X_1$'s influence, and vice versa)
• $X_2$ is uncorrelated, so $S_2 \approx T_2$ (as expected for an additive contribution)

Interpreting the Gaps¶

With independent inputs, the usual hierarchy holds: $S_i \leq \text{Shapley} \leq T_i$ — Shapley sits between first-order and total-order by distributing interaction variance.

With correlated inputs, this breaks down because:

• Correlation inflates $S_i$ (shared information through dependence)
• Correlation deflates $T_i$ (other variables proxy for $X_i$)
• Shapley effects remain interpretable: the Shapley axioms (efficiency, symmetry, dummy, additivity) still hold under correlation, making them the preferred single-number summary.

In both panels above:

• $X_2$ (uncorrelated, additive term $7\sin^2(x_2)$) shows $S_2 \approx \text{Shapley}_2 \approx T_2$ — no interactions
• $X_1$ and $X_3$ differ under correlation vs independence — the $\rho_{13}=0.8$ correlation transfers influence between them

Key Takeaways¶

1. MCShapley.compute() now returns Sobol indices in the same DataFrame — use method='exhaustive' for sobol_first and sobol_total columns.
2. Zero additional cost — Sobol indices are extracted from the same $v(u)$ values computed for Shapley effects.
3. Shapley sits between $S_i$ and $T_i$ — it fairly distributes interaction variance, making it interpretable as a single-number summary of each input's total importance.
4. The gaps reveal interactions — large $(T_i - S_i)$ indicates strong interactions; differences under correlation also reflect shared information between correlated inputs.