Marginal vs. Conditional Coverage: A 1-D Explainer

Parameters & Explainer Metrics

Calibration Size (n) 9 Reference samples

Confidence Target 80.0% Target coverage ($1 - \alpha$)

Boundary Index (k) 8 Order statistic threshold

Analytical Coverage 80.0% Theoretical minimum ($k / (n+1)$)

Calibration Size ($n$) 9

Significance Level ($\alpha$) 0.20

Wait, is Exact Conditional Coverage a Myth?

In a strict, distribution-free sense—yes, exact feature-conditional coverage is mathematically impossible to guarantee with finite samples. Standard conformal prediction makes a much humbler guarantee: Marginal Coverage. Understanding this distinction is the key to using conformal prediction correctly in the real world.

Marginal Coverage (Guaranteed) Averages over both the choice of your calibration set $\mathcal{D}_n$ and the new test sample. If you repeatedly collect new calibration data, the guarantee perfectly holds.

Conditional Coverage (Fluctuates) Conditions on your single, frozen calibration set. Once you collect data, the actual coverage of new test points will deviate from $1-\alpha$ due to the random placement of your samples!

Figure 1. 1D Coordinate Partition Explorer

Calibration points $\mathcal{D}_n$ (blue dots) partition the real line into $n+1$ slots. The shaded green region represents our prediction interval $[-\infty, Y_{(k)}]$ bounded by the evaluated quantile $Y_{(k)}$ (green dot). The test point is $Y_{n+1}$ (red dot).

Explainer Status

$$\text{Waiting to run simulation...}$$

Slot Frequency Density Breakdown

Under exchangeability, every single slot has a uniform $\frac{1}{n+1}$ probability of receiving the next sample. Observe how uniform the slots become when executing a Marginal simulation!

Partition Slot	Interval Bounds	Observed Counts	Empirical Frequency	Interval Status

Mathematical Foundation

Quantile Selection

$k = \lceil(1 - \alpha)(n+1)\rceil$

Inverse empirical ECDF slot matching rule

Coverage Guarantee

$\mathbb{P}(Y_{n+1} \le Y_{(k)}) \ge 1 - \alpha$

Distribution-free marginal coverage guarantee

Exchangeable Rank

$\mathbb{P}(\text{Rank} = i) = \frac{1}{n+1}$

Uniform distribution across slots