3D Computer Vision
by
Nassir Navab
exercises by
Darko Zikic,
Hauke Heibel, and
Martin Groher
4+2 SWS
Theoretische Informatik
Wahlpflichtfach
Announcements
- Invited Talk by Dr. Mirko Appel (Siemens Corporate Technology, München-Perlach):
Combined Orthographic and Perspective (COP) Images: Augmentation, Calibration & Reconstruction
- Exam
- Thursday, 10th of February, 5.00PM - 6.30PM (Hörsaal 1!!)
The exam will be partly closed, partly open book, so take your notes / textbooks with you. Moreover, you may use a basic calculator.
- Invited Talk by Prof. G. Hager (Johns Hopkins University, MD, USA):
Direct Methods for Visual Tracking: Basic Principles and Recent Progress
Abstract:
Over the last decade, direct methods for motion recovery have become widely used as a basis for visual tracking. In this talk, I will first review the basic principles of direct methods for motion tracking. Having done so, I will describe in some detail more recent work on direct methods for deformable stereo surfaces, and robotic tool tracking in endoscopic video. I will finally discuss a method of unifying many tracking methods, including direct methods, using kernel-based motion tracking methods.
- Thanks a lot to all of you who participated in the poll! The results can be viewed here.
Overview
Making a computer see was something that leading experts in the field of Artificial Intelligence thought to be at the level of difficulty of a summer student's project back in the sixties. Forty years later the task is still unsolved and seems formidable. A whole field, called Computer Vision, has emerged as a discipline in itself with strong connections to mathematics and computer science and looser connections to physics, the psychology of perception and the neuro sciences.
Over the past decade there has been a rapid development in the understanding and modelling of the geometry of multiple views in computer vision. The theory and practice have now reached a level of maturity where excellent results can be achieved for problems that were unsolved a decade ago, and often thought unsolvable. These tasks and algorithms include problems like:
Given two/three/multiple images, and no further information, compute/estimate:
- matches between the images
- the 3D position of the points that generate these matches
- the cameras that generate the images
adapted form Hartley & Zisserman's "Multiple View Geometry in Computer Vision"
This tasks and algorithms as well as the methods which allow us to reason about the quality of our results are the core of the lecture
3D Computer Vision.
Time & Location
- Lecture
- Tuesday 10:30AM-12:00 s.t. (MI 00.13.009A)
- Thursday 12:00-13:30PM s.t. (MI 00.13.009A)
- Exercise
- Thursday 3:15PM-4:45PM c.t. (MI 00.13.009A)
Topics
- Intro, motivation & Overview
- 2D Transformations
- Projective 2D Geometry
- 3D Transformations
- Projective 3D Geometry
- Camera Models
- Camera Calibration
- Single View Geometry
- Stereo Vision I – Image Rectification & Disparity maps
- Linear Algebra I - Singular Value Decomposition
- 3D reconstruction I: Factorization Method
- Epipolar Geometry : Essential & Fundamental Matrices
- Stereo Vision II – 3D Reconstruction
- Planes & Homographies
- Basics of Image Mosaicing
- Representation and Motion of Lines & Cylinders
- Three View Geometry of Lines & Cylinders
- The Trifocal Tensor
- 3D reconstruction II: Multi-view Reconstruction
- 3D reconstruction III: Motion and Structure from Motion
- Mixed Camera Models (Perspective and Orthographic)
- Combined Othrographic and Perspective (COP) Images: Calibration & Reconstruction
- 3D reconstruction IV: Fusion of 3D Maps (ICP)
- Image-based Rendering I
- Image-based Rendering II
- Conclusion & Discussions
Course Schedule
| Oct 19, 2004 | Intro & Motivation | |
| Oct 21, 2004 | Projective Geometry in 2D | Slides |
| Oct 26, 2004 | Projective Geometry in 3D / 1D | Slides |
| Oct 28, 2004 | Projective Geometry Summary / Conics and Quadrics | Slides |
| Nov 02, 2004 | Siemens Field Trip | |
| Nov 04, 2004 | 1st Intermediate Test, Multivariate Functions Reminder | Results of the Intermediate Test, Multivariate Calculus Reminder |
| Nov 09, 2004 | Parameter Estimation: Linear Estimation Methods (DLT) | Slides |
| Nov 11, 2004 | Parameter Estimation: Statistical error models | Slides |
| Nov 16, 2004 | No Class Today (SVV) | |
| Nov 18, 2004 | Parameter Estimation: Sampson Error, normalization | Slides |
| Nov 23, 2004 | Error Analysis | Slides |
| Nov 25, 2004 | Non-linear Parameter Estimation | Slides |
| Nov 30, 2004 | Camera Models | Slides |
| Dec 02, 2004 | No Lecture (Dies Academicus) | |
| Dec 07, 2004 | Camera Models II | Slides |
| Dec 09, 2004 | 2nd Intermediate Test (the not so easy one) | Results of the 2nd Intermediate Test |
| Dec 14, 2004 | Invited Talk: Direct Methods for Visual Tracking: Basic Principles and Recent Progress Prof. G. Hager (Johns Hopkins University, MD, USA) | Slides |
| Dec 16, 2004 | Computation of the Projection Matrix | Slides |
| Dec 21, 2004 | Epipolar Geometry | Slides |
| Dec 23, 2004 | no class: Merry Christmas!! | |
| Jan 11, 2004 | Epipolar Geometry II | Slides |
| Jan 13, 2004 | Epipolar Geometry III | Slides |
| Jan 18, 2004 | Epipolar Geometry IV | Slides |
| Jan 20, 2004 | Invited Talk: Combined Orthographic and Perspective (COP) Images: Augmentation, Calibration & Reconstruction Dr. Mirko Appel (Siemens Corporate Technology) | Slides |
| Jan 25, 2004 | The Trifocal Tensor I, Plücker Line Coordinates | Slides |
| Jan 27, 2004 | The Trifocal Tensor II, Cylinders | Slides , Slides on Cylinders |
| Feb 01, 2004 | The Trifocal Tensor III | Slides |
| Feb 03, 2004 | | |
| Feb 08, 2004 | | |
| Feb 10, 2004 | | |
Exercises
| Oct 21, 2004 | MatLab, Homogeneous Coordinates | Darko Zikic & Martin Groher | Exercise Sheet 1 | MatLab Worksheet , 2D Projective Space Handout , Conics & Quadrics Basics | Solution for Exercise Sheet 1 |
| Oct 28, 2004 | Transformations, Conics, Quadrics | Darko Zikic & Martin Groher | Exercise Sheet 2 | MatLab Worksheet | Solution for Exercise Sheet 2 |
| Nov 4, 2004 | Multivariate Functions, Affine Rectification, SVD | Darko Zikic & Martin Groher | Exercise Sheet 3 | SVD Basics | |
| Nov 11, 2004 | Parameter Estimation, Error Models | Martin Groher | Exercise Sheet 4 | | Solution for Exercise Sheet 4 |
| Nov 18, 2004 | Covariance, Normalization of DLT, Mosaicing | Darko Zikic | Exercise Sheet 5 | Probability and Statistics Reminder, nice example for mosaicing: Spiegel Online Article | |
| Nov 25, 2004 | Nonlinear Estimation and Error Propagation | Martin Groher | Exercise Sheet 6 | JacobianReprojection.pdf | Solution for Exercise Sheet 6 |
| Dec 2, 2004 | no exercises (Dies Academicus) | | | | |
| Dec 9, 2004 | Invited Talk: ARToolkit (Martin Bauer), Camera Models / Calibration | Martin Groher | Exercise Sheet 7 | exercise06e3.m | Solution for Exercise Sheet 7 |
| Dec 16, 2004 | Camera Calibration | Martin Groher | Exercise Sheet 8 | Paper of Zhengyou Zhang | Solution to Exercise Sheet 8 |
| Dec 23, 2004 | no exercises: Merry Christmas! | | | | |
| Jan 13, 2004 | Camera Calibration II, Radial Lens Distortion | Martin Groher | Exercise Sheet 9 | | Solution to Exercise Sheet 9 |
| Jan 20, 2004 | Mono- and Stereo Image Rectification | Hauke Heibel | | Slides | |
| Jan 27, 2004 | The Absolute Conic | Martin Groher | Exercise Sheet 10 | | |
| Feb 3, 2004 | | | | | |
| Feb 10, 2004 | | | | | |
Programming Assignment
- Pix
- here are some test images. some are more suitable than others. but all quite boring. so feel encouraged to make your own images. the cooler the better...
Readings
- Primary Readings
- Multiple View Geometry in Computer Vision; by Richard Hartley,
Andrew Zisserman
- General Introduction to 3D Computer Vision
- Three-Dimensional Computer Vision; by Olivier Faugeras
- Computer Vision: A Modern Approach; by David A. Forsyth & Jean Ponce
- Introductory Techniques for 3-D Computer Vision; by Emanuele Trucco & Alessandro Verri
- More Specific Readings
- The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene and Some of Their Applications; by Olivier Faugeras, Quang-Tuan Luong, Theodore H. Papadopoullos; MIT Press; 2001