Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

THIS WEBPAGE IS DEPRECATED - please visit our new website


Tracking and Detection in Computer Vision - Winter Term 2018/19

Administrative Info

Lecture by Dr. Slobodan Ilic
Exercises by: Haowen Deng, Sergey Zakharov, Mai Bui

Type: Lecture Module IN2210
Programs: Informatics (Bachelor, Master)
Biomedical Computing (Master)
Robotics, Cognition, Intelligence(Master)
SWS: 2+4
ECTS: 7 Credits
Course Language: English

Time, Location & Requirements

Lectures: From 22.10.18 at 14:00-16:00h at 00.13.009A .

Exercises: From 25.10.18 at 14:00-16:00h at 00.13.009A.


  • The classes and exams are in English.
  • For the midterm exam, nothing but your pens and your calculators are allowed.
  • For the final exam, nothing but your pens, your calculators and one DIN A4 page (handwritten, front and back side) with notes are allowed.
  • There is no repetition exam.



Computer Vision has emerged as a key discipline in computer science. This is not only evident by a growing and highly competitive research community with a high impact factor in computer science, but also by the emergence of numerous vision companies turning research ideas into a myriad of commercial applications. Besides well-known studies of 3D geometry and camera models, object tracking and detection in images and videos becomes one of the principal research directions of modern Computer Vision. The main objective of this course is to provide students with a gradual introduction to modern tracking, detection and recognition techniques developed in the last years. The course will provide in-depth knowledge of image features, their detection and description, matching techniques, key-point recognition, basic and advanced tracking algorithms based on image features and image intensities. The course will address both hand crafted features and learned features with deep learning approaches. So it will be mixture of established ane emerging methods for object detection, recognition, pose estimation and tracking.

In the end, the students will have a thorough description of the most important tracking and detection techniques. They should be able to understand and implement those solutions and apply them in reasonably complex problems. The concepts described in this course will be accompanied with brief explanations of the necessary mathematical tools. The participants of this course will be given an important basis to follow the vast and growing Computer Vision literature and use the acquired knowledge to solve new practical problems.

Who should attend this course?

Bachelor students in their last years and all Master students that are interested in learning about the modern and well-established concepts and algorithms related to tracking, detection and recognition in Computer Vision, or desire to use those techniques in their research.

Prerequisites: Most of the knowledge required should be part of the normal background in Computer Science, undergraduate/graduate Mathematics and Geometry.


The mid-term exam is fixed for: 29.11.18 from 18:30 until 20:00 in Room MW 2001 .

The final exam is preliminary date is: 01.03.18 from 15:30 until 17:00 in room MW 0001.

The exercise grading sessions will be held in parallel in the following rooms:

  • 22.11.18: 00.13.009A, 03.13.010, 01.13.010
  • 6.12.18: 00.13.009A, 03.13.010, 01.13.010
  • 13.12.18: 00.13.009A, 00.11.038, 00.12.019
  • 17.1.19: 00.13.009A, 03.13.010, 01.13.010
  • 7.2.19: 00.13.009A, 00.11.038, 01.13.010

Exercise projects

The project will be done in MATLAB, C++ and Python and students are expected to do them on their own computer/laptop. They will consist of three projects that will include one or several topics presented at the lectures. The projects will be presented to the students during the exercise sessions, where they will be instructed how to do the projects. Each project is divided into a number of exercises and each of them corresponds to an actual homework which is graded. In total all home work exercises result in a bonus for the final exam. The deadlines for the presentations and submissions of the results of the homeworks will be provided on this website. During the regular exercise classes student can work on the project and ask questions to the tutors.

The exact due dates for each project are announced next to the exercise description on this website and will be defined beforehand (check exercise section). Students have to make groups of up to FOUR students and send the names to tdcv.tum@gmail.com. After that, those who didn't create groups will be associated to groups based on a random selection and will receive a notification about this. If you are looking for a (TDCV) partner tell us and we will assign you one!

During the the presentation of the results for each project each group will have 5 min to present their solution in terms of slides, and also run the code live. This will be followed by questions from the tutors (~ 5min). All members of the group will be questioned about the algorithms and their implementation, so only those who can prove that they understood both theory and practice will get the respective points. Thus, having a working exercise doesn't mean that everyone in the group is going to get full points. Please note that only exercises which work might get the maximal number of points. There will be no "debugging-correcting"-sessions for your solutions during the project results presentation! If you have questions, ask them beforehand during the exercise sessions.

All homework should be submitted through e-mail before the exercise session with the following format and information.

  1. To: tdcv.tum@gmail.com
  2. Subject: [TDCV17] Exer<exercise number> - <group number> - <last names of all members>
    For example: [TDCV17] Exer1 - Group42 - Huang, Tan
  3. Attachment: The whole source code for this homework.
  4. Body: This should include the (a) full names and Matrikel-Nr. of each member; and, (b) reference to any code used in the homework that is not specified on the exercise sheet or by the tutors.

You can also find a short MATLAB tutorial here.


You are required to register via TUMOnline!


For some basic concepts the textbook "Computer Vision: a modern approach" by David Forsyth and Jean Ponce can be used. Also, most topics can be found in a new book of Rick Szeliski from Microsoft Research, available also on-line. For machine learning topics it is recommended to use the book of Christopher Bishop, Pattern Recognition and Machine Learning. And for deep learning, the Stanford course on CNNs available on-line. . However, since the majority of the topics is state-of-the-art, students will be referred to the relevant publications which will be provided during the course and can be regarded as additional lecture notes.

Lecture Schedule and Content

Date Topic Conducted by Material/Literature
22.10. Introduction Dr. S. Ilic Introduction
29.10. Camera models, Model based tracking, Pose estimation from 2D-3D coresspondencies (DLT, P-n-P), Rotation parametrization Dr. S. Ilic Camera models and pose estimation
Related paper:
1. Monocular Model-Based 3D Tracking of Rigid Objects: A Survey.
05.11. Non-linear optimization, Robust estimators, RANSAC Dr. S. Ilic Original: Non-linear optimization, Robust estimators, Ransac
Annotated at the lecture: Non-linear optimization, Robust estimators, Ransac
12.11. Convolution, Non-linear filtering, Gaussian filtering, Image derivatives, Edge detection Dr. S. Ilic Filtering and Edge Detection
19.11. Local invariant feature detectors(Harris corners, FAST etc.) and feature descriptors(Difference of Gaussians and SIFT, SURF, HOG) Dr. S. Ilic Feature Detection
Related papers:
1. Feature detectors
2. Harris-Laplace
Feature descriptors
26.11. Keypoint recognition: Randomized and regression trees, FERNS, Hough forests Dr. S. Ilic Keypoint recognition
29.11. Midterm exam Dr. S. Ilic Midterm exam
Solutions midterm exam
Grades midterm exam
03.12. Feature learning and Classification with Deep Neural Networks Dr. S. Ilic Deep Neural Networks annotated
Deep Neural Networks
10.12. Haar features; Integral images; Ada-Boost; Viola-Jon Dr. S. Ilic
17.12. Object detection with deep learning (R-CNN, Fast-RCNN, SSD) Dr. S. Ilic Object Detection with ConvNets
07.01. Object detection with deep learning (YOLO, SSD, SSD-6D, Triplet Loss)) Dr. S. Ilic
14.01. Template matching Dr. S. Ilic
21.01. TBD Dr. S. Ilic  
28.01. TBD    
04.02. TBD Dr. S. Ilic  

Exercise Projects Schedule

If you have questions about a specific exercise, please direct them to the responsible tutor. At the due date the results of the projects will be presented by each group of students followed by the questions of the tutors. The students must send the code min 1h before the due date of the presentation of their project results. The students should be ready to run the code after their presentation as well as show parts of the code and be prepared to answer the questions about their implementation.

Date Topic Conducted by Material/Literature Sample Output
25.10.2018 MATLAB installation and introduction about the basics Mai Bui , Ivan Sugurov and Roman Kaskman Info Preparation for Project1 -
08.11.2018 Project 1: Keypoint based object detection, pose estimation and refinement Exercise 1. Model preparation Mai Bui , Ivan Sugurov and Roman Kaskman Exercise 1 Code  
15.11.2018 Project 1: Keypoint based object detection, pose estimation and refinement Exercise 2. Pose Estimation with PnP? and RANSAC Mai Bui , Ivan Sugurov and Roman Kaskman Exercise 2 Code  
22.11.2018 Exercise 1 - Homework presentation and grading All    
29.11.2018 Project 1: Keypoint based object detection, pose estimation and refinement Exercise 3. Pose refinement with non-linear optimization with LM. Mai Bui , Ivan Sugurov and Roman Kaskman Exercise 3 Data  
06.12.2018 Exercise 2 - Homework presentation and grading All    
13.12.2018 Project 2: Object classificartion with random forests Exercise 3 - Homework presentation and grading Haowen Deng , and Mehdi Hamad + All Project 2 Data
20.12.2018 Project 2: Object classificartion with random forests Haowen Deng , and Mehdi Hamad    
10.01.2018 Project 3: Object recognition and pose estimation with Deep Learning Sergey Zakharov , and Onur Eken Project 3 Data
17.01.2018 Project 2 - Homework presentation and grading All    
24.01.2018 Project 3: Q&A session Sergey Zakharov , and Onur Eken    
07.02.2018 Project 3 - Homework presentation and grading . All    

Title: Tracking and Detection in Computer Vision
Professor: Dr. Slobodan Ilic
Tutors: Tolga Birdal, Haowen Deng, Sergey Zakharov, Mai Bui
Type: Lecture
Information: 2 + 4; 7 ECTS
Term: 2018WiSe

Edit | Attach | Refresh | Diffs | More | Revision r1.42 - 12 Apr 2019 - 07:36 - MaiBui

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif