PointNet

Definition

A point-based network consumes the points directly and needs permutation invariance, a variable number of points and locality awareness.

PointNet (Qi et al., CVPR 2017)

$$f(g(h({\mathbf{x}}_1),\dots ,h({\mathbf{x}}_n))),g=\max \text{ (channel-wise)},h,f=\text{MLPs}$$

The shared $h$ is equivariant; the symmetric $\max$ makes it invariant. T-Nets ($3\times 3$, $64\times 64$) undo rotations, regularized with $\Vert I-A{A}^T{\Vert }_F^2$. Segmentation: concatenate local (64) and global (1024) features → $n\times 1088$.

	PointNet	PointNet++	Point Transformer
Key idea	per-point MLP + max pooling	set abstraction: FPS, grouping, PointNet per group	vector attention over kNN + relative PE
Solves	permutation invariance	locality	expressive local aggregation
Segmentation	concat local + global	feature propagation (interpolation + skip links)	U-Net with transition up (interpolation + sum)

Trap

Sorting the points is not a solution: a tiny perturbation changes the order drastically.

Appears in

• Lecture 6.2, PointNet
• Lecture 6.2, PointNet++
• Lecture 6.2, Point Transformer
• Lecture 1, course overview
• Lecture 11, 3DShape2VecSet: farthest point sampling for latent queries

Related concepts

• Point Cloud
• Attention
• Voxel Grid