Master's thesis

Supervision by: Iro Laina, Keisuke Tateno, Federico Tombari
Thesis by: Helisa Dhamo

Abstract

Structured light sensors consisting of an infrared camera and projector are widely used in real-time applications to generate depth maps. However, the obtained depth maps sometimes lack accuracy. Recently, random forests were deployed to solve this problem by learning how to estimate the depth from a projected infra-red pattern [1]. This master's thesis aims at studying deep learning approaches to predict depth maps from infrared images with projected patterns. For this purpose, we will start with a Fully Convolutional Residual Neural Network (FCRN) [2] that learns to regress a depth map from a single RGB image. The first part of the thesis will be providing a tool for transcribing architectures (in particular FCRN) between deep learning frameworks, namely from MatConvNet^? to TensorFlow^?. The second part will focus on learning depth maps from single IR images instead. Further modifications will be developed, aiming to surpass the accuracy of standard consumer depth cameras in real-time execution.

Resources

[1] Ryan Fanello, S., Rhemann, C., Tankovich, V., Kowdle, A., Orts Escolano, S., Kim, D., & Izadi, S. (2016). Hyperdepth: Learning depth from structured light without matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5441-5450).
[2] Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016, October). Deeper depth prediction with fully convolutional residual networks. In 3D Vision (3DV), 2016 Fourth International Conference on (pp. 239-248). IEEE.