DaDepthFromStructLight

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Master's thesis

Supervision by: Iro Laina, Keisuke Tateno, Federico Tombari
Thesis by: Helisa Dhamo

Abstract

Structured light sensors consisting of an infrared camera and projector are widely used in real-time applications to generate depth maps. However, the obtained depth maps sometimes lack accuracy. Recently, random forests were deployed to solve this problem by learning how to estimate the depth from a projected infra-red pattern [1]. This master's thesis aims at studying deep learning approaches to predict depth maps from infrared images with projected patterns. For this purpose, we will start with a Fully Convolutional Residual Neural Network (FCRN) [2] that learns to regress a depth map from a single RGB image. The first part of the thesis will be providing a tool for transcribing architectures (in particular FCRN) between deep learning frameworks, namely from MatConvNet? to TensorFlow?. The second part will focus on learning depth maps from single IR images instead. Further modifications will be developed, aiming to surpass the accuracy of standard consumer depth cameras in real-time execution.

Resources

[1] Ryan Fanello, S., Rhemann, C., Tankovich, V., Kowdle, A., Orts Escolano, S., Kim, D., & Izadi, S. (2016). Hyperdepth: Learning depth from structured light without matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5441-5450).
[2] Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016, October). Deeper depth prediction with fully convolutional residual networks. In 3D Vision (3DV), 2016 Fourth International Conference on (pp. 239-248). IEEE.

Students.ProjectForm
Title: Depth Prediction from Structured Light using Fully Convolutional Neural Networks
Abstract: Structured light sensors consisting of an infrared camera and projector are widely used in real-time applications to generate depth maps. However, the obtained depth maps sometimes lack accuracy. Recently, random forests were deployed to solve this problem by learning how to estimate the depth from a projected infra-red pattern [1]. This master's thesis aims at studying deep learning approaches to predict depth maps from infrared images with projected patterns. For this purpose, we will start with a Fully Convolutional Residual Neural Network (FCRN) [2] that learns to regress a depth map from a single RGB image. The first part of the thesis will be providing a tool for transcribing architectures (in particular FCRN) between deep learning frameworks, namely from MatConvNet? to TensorFlow?. The second part will focus on learning depth maps from single IR images instead. Further modifications will be developed, aiming to surpass the accuracy of standard consumer depth cameras in real-time execution.
Student: Helisa Dhamo
Director: Prof. Nassir Navab
Supervisor: Iro Laina, Keisuke Tateno, Federico Tombari
Type: DA/MA/BA
Area: Computer Vision
Status: finished
Start: 01.12.2016
Finish: 31.05.2017
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.2 - 16 Jan 2018 - 15:53 - IroLaina