Point Map

Definition

A point map $\mathbf{X}\in {\mathbb{R}}^{W\times H\times 3}$ is a dense, pixel-aligned 3D point cloud: a 1-to-1 mapping ${\mathbf{I}}_{i,j}\leftrightarrow {\mathbf{X}}_{i,j}$ between image pixels and 3D scene points, expressed in one camera’s frame. It is more structured than a point cloud, and converts easily to a depth map ($z$ channel).

	Point cloud	Point map	Depth map
Structure	unordered set	image grid, $(x,y,z)$ per pixel	image grid, $z$ per pixel
Pixel ↔ 3D	none	given	needs intrinsics

What you can read from a point map: depth ($z$), pixel correspondences (mutual nearest neighbors in 3D), focal length (reprojection with a centered principal point), relative pose (Procrustes, or PnP + RANSAC via the 2D-3D correspondences).

Model	Output
DUSt3R (2024)	2 point maps in camera 1’s frame + confidence, from 2 unposed images; global alignment for more views
MASt3R (2024)	metric point maps + matching features
Fast3R, VGGT (2025)	many views in one forward pass; VGGT also cameras, depth and tracks

Trap

DUSt3R’s global alignment minimizes a 3D error between point maps, not the 2D reprojection error of bundle adjustment.

Appears in

• Lecture 9, Point Maps
• Lecture 9, DUSt3R
• Lecture 9, Downstream tasks
• Lecture 9, VGGT
• Lecture 4, VGGT teaser
• Lecture 1, course overview

Related concepts

• Point Cloud
• Structure from Motion
• Monocular Depth Estimation
• Attention