Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Semantic Simultaneous Localization And Mapping


This thesis aims to leverage 2D (or 3D) semantic information in a simultaneous localization and mapping (SLAM) system to handle typical difficulties, such as reconstructing/filtering moving objects.

The project can be split into two parts. The first one is mostly related to 3D reconstruction and the other one is related to tracking:

(1) 3D scene reconstruction often suffers from dynamic objects which lead to tracking failure and blurry reconstruction. Fusion++[1] proposed an object-level SLAM system which uses semantic information to reconstruct a map with the knowledge of objects. MaskFusion?[2] on the other hand focuses on reconstructing moving 3D objects individually. The goal of this thesis is to combine the two methods above, to enable the reconstruction of a dynamic scene.

(2) The main goal of this part is to design a semantic tracking system based on 2D (or 3D) semantic information to improve the robustness of existing tracking methods. For a frame-to-model ICP tracking system [3], a huge amount of drifting occurs when the observed geometry is mostly uniform. With the help of semantic information, it is possible to maintain the performane of an ICP tracker through these difficult regions. For feature-based tracking systems, semantic information can reduce the number of incorrectly matched points in the matching process for points belonging to different classes.


Very good C++
Computer vision and multiview geometry


[1]Fusion++: Volumetric object-level slam

[2]Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects

[3]KinectFusion: Real-time dense surface mapping and tracking

Title: Semantic Simultaneous Localization And Mapping
Director: Federico Tombari
Supervisor: Shun-Cheng Wu
Type: Master Thesis
Area: Computer Vision
Status: open
Start: 1 April 2020
Thesis (optional):  

Edit | Attach | Refresh | Diffs | More | Revision r1.4 - 19 Feb 2020 - 15:23 - NikolasBrasch