MaPhotoRealisticReconstruction

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Improving photometric quality of SLAM

Abstract

We offer a Master Thesis in the topic of 3D computer vision and deep learning, in particular on real-time 3D reconstruction from a sequence of frames. Existing SLAM approaches rely on different fusion methods to integrate sensor data from different view angles in order to reconstruct a scene. For example, KinectFusion?[1] uses running average on TSDF[3] and RGB values on each voxel. Alternative fusion methods were proposed [2]. These methods reconstruct the geometry accurately, nevertheless the reconstructed texture is usually blurry and and missing fine details (See Figure1).

In this project, we aim to develop a novel approach that is able to reconstruct a more photorealistic scene in real-time. We are looking for a motivated student with prior experience in deep learning and computer vision, and good knowledge of Python and C++. The project will be carried out in cooperation with Google Zurich.

Requirements

  • Prior experience (theoretical and practical) of computer vision and deep learning.
  • Knowledge of SLAM methods.
  • Good knowledge of Pytorch
  • Good coding skills in python and C++

Literature

  • [1] Izadi, Shahram, et al. "KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera." Proceedings of the 24th annual ACM symposium on User interface software and technology. 2011.
  • [2] Keller, Maik, et al. "Real-time 3d reconstruction in dynamic scenes using point-based fusion." 2013 International Conference on 3D Vision-3DV 2013. IEEE, 2013.
  • [3] S. Osher and R. Fedkiw. Level set methods and dynamic implicit surfaces. Springer, 2002.

Application

If you are interested, please contact me via e-mail:

robot_mirror

Figure 1. Left. Rendered view from a reconstructed scene. Right. The input RGB used to reconstruct the scene.


Students.ProjectForm
Title: Improving photometric quality of SLAM
Abstract: Existing incremental scene reconstruction approaches rely on different fusion methods to integrate sensor data from different view angles in order to reconstruct a scene. For example, KinectFusion?[1] uses running average on TSDF[3] and RGB values on each voxel. Similar aggremetion methods are also used in other works[2]. Accurate geometry is possible to be reconstructed by using this approach. However, the reconstructed texture is usually blurry and is less realistic (See Figure1).
Student:  
Director: Federico Tombari
Supervisor: Shun-Cheng Wu
Type: DA/MA/BA
Area: Computer Vision
Status: open
Start:  
Finish:  
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.7 - 30 Jan 2021 - 10:11 - ShunChengWu