Notes for the master course Hands-on AI based 3D Vision (Prof. Dr.-Ing. Gerard Pons-Moll, University of Tübingen / MPI for Informatics, Summer Semester 2026).
The course is about understanding and reconstructing the 3D world from images and video. It starts with the fundamentals of multiple view geometry and goes up to recent methods: NeRF, Gaussian Splatting, 3D diffusion models and token-based reconstruction.
Goal of the course
After this course, students should be able to:
- understand research papers related to 3D vision and assess how they fit within the state of the art
- develop classical and modern algorithms: camera parameter estimation, Structure-from-Motion, point cloud processing, neural rendering techniques, generative models
Lectures
| # | Lecture | Key concepts |
|---|---|---|
| 1 | Introduction to 3D Computer Vision | 3D vision vs. computer graphics, applications, history of 3D vision |
| 2.1 | Image Formation | pinhole camera, lens camera, intrinsics, extrinsics, projection |
| 2.2 | Rotations | rotation matrices, Euler angles, gimbal lock, quaternions, axis-angle, exponential map |
| 3 | Classical Reconstruction Methods | ”inverting” image formation, epipolar geometry, essential matrix, 8-point algorithm, triangulation, Structure-from-Motion, bundle adjustment |
| 4 | Stereo Vision and Depth Estimation | epipolar geometry, rectification, disparity, stereo matching, multi-view stereo, monocular depth |
| 5 | Surface Reconstruction | implicit vs. explicit geometry, SDF, marching cubes, point clouds, Poisson reconstruction, Procrustes, ICP |
| 6.1 | Neural Fields | voxel grids, octrees, neural fields, triplanes, hash encoding, neural implicits |
| 6.2 | Point Cloud Processing | PointNet, permutation invariance, PointNet++, point transformers |
| 7 | Neural Radiance Fields (NeRF) | volume rendering, transmittance, positional encoding, NeRF variants |
| 8 | 3D Gaussian Splatting | 3D Gaussians, covariance, splatting, alpha blending, adaptive density control |
| 9 | Learning-Based 3D Reconstruction | COLMAP, bundle adjustment, DUSt3R, point maps, 3D foundation models |
| 10 | Diffusion Models | GAN, VAE, DDPM, DDIM, Stable Diffusion, multi-view diffusion |
| 11 | Flow Matching and 3D Generation | flow matching, score distillation sampling, point cloud diffusion, TRELLIS, Hunyuan3D |
Lecture pages are linked here as soon as they are written.
Terms that appear in several lectures have their own short page under Concepts.
How to read these notes
Every lecture page has the same structure: a short TL;DR, the content in the order of the slides, a self-test and links to related lectures. The table of contents on the right follows the slide sections. Small labels like Slides 12-15 say where a topic is in the original slides.
Callouts mark what matters most:
Definition
The exact meaning of a term.
Key formula
Formulas you should know by heart.
Intuition
Why something works, in plain words.
Common mistake
Typical misunderstandings and wrong exam answers.
Exam relevance
What was asked in the exam or is likely to be asked.
Self-test question
Answer
Click to reveal.
Course organization
| Lecturer | Prof. Dr.-Ing. Gerard Pons-Moll |
| Teaching assistants | Niklas Berndt, Pradyumna YM, Yannan He |
| Workload | 6 ECTS, one lecture + one tutorial per week |
| Grading | 50% exam, 20% exercises, 30% project |
| Course page | https://virtualhumans.mpi-inf.mpg.de/3DVision26/ |
- Exercises consist of theoretical questions and programming exercises, done in teams of 2 or 3 people.
- Project: a mini research project (~6 weeks). It is graded on the report (motivation, prior work, methodology, evaluation, limitations), on how substantial the method is, on the robustness of the result and on the presentation.
- To pass the course, you need to pass the exercises, the project and the exam.
Requirements: linear algebra, probability theory, machine learning, optimization and programming skills (Python). PyTorch and computer graphics background are a plus.