Cost Volume

Definition

A cost volume stores, for every pixel of the reference image and every hypothesis (disparity or depth), how well the views agree. Classical: height × width × number of disparities, with one cost per entry. Learned: a feature vector per entry, processed with 3D convolutions.

Stereo vs. multi-view

	GC-Net (rectified stereo)	MVSNet (multi-view)
Hypothesis	disparity $d\in \{0,\dots ,D_{max}-1\}$	depth $z\in \{z_1,\dots ,z_D\}$
Align views by	horizontal shift	differentiable homography (plane sweep)
Combine	concatenate, $2F$ channels, 2 views	variance, $F$ channels, any $N$
Shape	$H\times W\times D_{max}\times 2F$	$H\times W\times D\times F$

Intuition

Always the same pattern: set up hypotheses, measure consistency, pick the best. Stereo hypothesizes 1D shifts along a scanline; multi-view hypothesizes planes in 3D, because the cameras have arbitrary poses.

Appears in

• Lecture 4, Cost Volume
• Lecture 4, GC-Net
• Lecture 4, MVSNet

Related concepts

• Stereo Matching
• Multi-View Stereo
• Disparity