Persistent SLAM

Introduction

The development of scene reconstruction and understanding has achieved remarkable results. An indoor scene can be easily reconstructed with a consumer depth sensor. A reconstructed scene can also be processed to semantically understand the objects inside the scene. However, very few methods try to reuse the knowledge from previous scans. Reusing previous scans is a challenging task since it involves retrieving and registering the scenes that have potential changes (See Fig. 1).

In this project, we aim to develop methods that focus on reusing the knowledge from previous scans. The target goal includes: a) improve reconstruction quality, b) retrieve scenes, c) relocalize camera pose, and d) re-identify objects and changes.

This project is offered as a Master Thesis or Guided Research. In addition, a Hiwi position could be paired to this research project

Objective

The goal of this project is to develop methods to reuse the knowledge from previous scans. The possible direction can be:

• retrieve scans
• identify changes
• reuse previous knowledge

Requirement

• Knowledge in Python and C++
• Knowledge in Deep learning
• Knowledge in Computer vision and 3D Geometry

Literature

• [1] J. Wald, A. Avetisyan, N. Navab, F. Tombari* and M. Niessner*, "RIO: 3D Object Instance Re-Localization in Changing Indoor Environments", Oral, ICCV 2019.
• [2] J. Wald*, H. Dhamo*, N. Navab, and F. Tombari. "Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions", CVPR 2020.
• [3] J. Wald, T. Sattler, S. Golodetz, T. Cavallari and F. Tombari. "Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes", ECCV 2020.
• [4] S. Wu, J. Wald, K. Tateno, N. Navab and F. Tombari. "SceneGraphFusion: Incremental 3DScene Graph Prediction from RGB-D Sequences", CVPR 2021.

Fig.1 Two scans of the same scene taken at a different time with semantics.