Lab Course / Praktikum: Advanced Topics in 3D Computer Vision

by Prof. Nassir Navab
Team: Benjamin Busam, Mahdi Saleh, Patrick Ruhkamp, Hyun Jun Jung
Type: Master Practical Course Module IN2106
SWS: 6
Programs: Informatics (Master), Biomedical Computing (Master)
ECTS: 10 Credits

Preliminary Meeting ("Vorbesprechung"): Tuesday, 14.07.2020 at 1.00 pm. Zoom Meeting
Further Meetings: See the schedule below. According to the pandemic situation we will partly or fully have our meetings / lectures via zoom.

This course in TUMOnline
The course will be conducted in English.

Announcements

• If you want to register for the course, please provide your motivation and current Transcript of Records to the course organizers: , , ,
• Application Deadline: 21-JUL-2020
• Preliminary Meeting: 14-JUL-2020

Registration

For the application, please use the following form (Deadline : 21.07.2020): Application Form
Please also register in the TUM matching system for the course registration. Keep in mind that your chances to be assigned to the course increase if you give it a higher rank in your choices. For further details about how the matching system works and its schedule please check this website.
We select appropriate candidates based on their background, interests, and motivation.

General Content

This course focuses on different recent topics in 3D computer vision and their application. The first set of lectures on specific vision problems (such as 6D Pose Estimation, Structure from Motion, Deep Learning) lays the foundation for practical challenges carried out by the student individually. While working on these problems the student becomes familiar with these topics and decides for a direction for the group phase. In the group phase, a group of students is supported by coaches from both industry and academia. The group identifies a particular 3D vision problem in one of these areas and develops a prototype to solve it. The prototype solution is finally presented with a live demonstration.

Example Applications

Example applications to follow.

Prerequisites

• Basic knowledge with C++ and Python programming
• Experience with basic computer vision and machine learning concepts
• Enthusiasm to tackle challenging problems
• Passion to create something new with a team

Passing Requirements

• Active participation in all mandatory workshops and meetings
• Submission of all individual challenge solutions in time
• Final group presentation with live prototype demonstration

Schedule

The schedule is preliminary and still subject to change. Please check the course website regularly!

Date	Time	Location	Topic	Conducted by	Materials
14.07.2020	13.00 - 14.00	03.13.010 + Zoom	Preliminary Meeting via Zoom	Patrick Ruhkamp Benjamin Busam Hyun Jun Jung	Preliminary Meeting Slides
03.11.2020	14.00 - 15.30	03.13.010 + Zoom	Lecture 1: Basic Tips and Knowledge for Deep 3D Vision	Hyun Jun Jung	Course Organization Lecture Slides Lecture Video Challenge 01
09.11.2020	23:59 CET	Upload	Solutions to Challenge 1	Students
10.11.2020	14.00 - 15.30	Zoom	Lecture 2: Sensing Depth with 3D Computer Vision	Benjamin Busam	Lecture Slides Lecture Video Challenge 02 Example Solution
16.11.2020	23:59 CET	Upload	Solutions to Challenge 2	Students
17.11.2020	14.00 - 15.30	Zoom	Lecture 3: Visual Self-Localization - Classical Methods and Learning-based Approaches	Patrick Ruhkamp	Lecture Slides Lecture Video Challenge 03 Challenge 03 PGO
23.11.2020	23:59 CET	Upload	Solutions to Challenge 3	Students
24.11.2020	14.00 - 15.30	Zoom	Lecture 4: 3D Deep Learning	Mahdi Saleh	Lecture Slides Lecture Video Challenge 04 Workshop Info
30.11.2020	23:59 CET	Upload	Solutions to Challenge 4	Students
05.12.2020	10.00 - 14.00	Zoom	Project Brainstorming Workshop (attendance mandatory)	Benjamin Busam Mahdi Saleh Patrick Ruhkamp Hyun Jun Jung	Workshop Intro Presentation Template
12.01.2021	14.00 - 15.30	Zoom	Project Update Presentations (attendance mandatory)	Benjamin Busam Mahdi Saleh Patrick Ruhkamp Hyun Jun Jung
11.02.2021	18.00 - ca. 20.30	Zoom	Final Workshop (attendance mandatory)	Benjamin Busam Mahdi Saleh Patrick Ruhkamp Hyun Jun Jung

Project Proposals

Project ID	Title	Supervisor	Students	Materials
#1	Object SLAM	Patrick Ruhkamp	Andy, Ezgi, Leander	Project Object-SLAM
#2	3D Object detection and Bounding Box Refinement	Mahdi Saleh	Lukas, Yusuf	Project 3D-BB
#3	Temporally Consistent Video Depth	Benjamin Busam	Daoyi, Hanzhi	Project Depth
#4	Multi-modal depth estimation/improvement	Hyun Jun Jung	Julia, Thomas, Jingpei	Project Depth 2

Coding

• Online C++ API reference
• Python 3.6 documentation
• OpenCV Documentation
• PyTorch documentation
• CMake
• MS Visual Studio (TUM StudiSoft)

Versioning

• Github, a freeware Github client for Mac/Windows
• Github tutorial for beginners
• With TUM account: LRZ Gitlab