Flow Matching

Formula

Sampling: $\frac{d}{dt}{X}_t=u_t^{\theta }(X_t)$, from $X_0\sim p$ (noise) to $X_1\sim q$ (data).

Training (linear path): $X_t=(1-t)X_0+t{X}_1$,

$${\mathcal{L}}_{CFM}(\theta )={\mathbb{E}}_{t,X_0,X_1}{\Vert u_t^{\theta }(X_t)-(X_1-X_0)\Vert }^2,{\nabla }_{\theta }{\mathcal{L}}_{FM}={\nabla }_{\theta }{\mathcal{L}}_{CFM}$$

Intuition

Velocities are linear: the marginal velocity is the average of simple per-pair (conditional) velocities and generates the marginal probability path. So random noise-data pairs are enough.

	Diffusion	Flow matching
Path	stochastic SDE	deterministic ODE
Target	noise / score	velocity
Steps	thousands	10-100

For a Gaussian source both are equivalent up to reparameterization.

Uses in 3D generation: TRELLIS generates structured latents (SLAT, features at active voxels) with flow transformers; Hunyuan3D-DiT generates unstructured latents (VecSets) with a flow-based diffusion transformer. SDS (DreamFusion) instead distills a frozen 2D diffusion model into a 3D representation by optimization.

Appears in

• Lecture 11, Flow Matching
• Lecture 11, FM vs. diffusion
• Lecture 11, Structured vs. unstructured latents
• Lecture 11, TRELLIS
• Lecture 11, SDS

Related concepts

• Diffusion Model
• Neural Field
• Gaussian Splatting