ProjectDepthPrediction

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Depth Estimation from a Single RGB Image

Depth Estimation from a Single RGB Image

Scientific Director: Nassir Navab

Contact Person(s): Iro Laina, Christian Rupprecht

Keywords: Computer Vision

Abstract

Depth estimation from multiple views (stereo vision) or other single-view assumptions (motion, shading, defocus) has been well studied in literature. However, estimating the depth map of a scene from a single RGB image remains an open problem due to the inherent ambiguity of mapping color/intensities to depth values, i.e. a 2D image could correspond to multiple 3D world scenarios. Within the spectrum of this project, we address this problem using a deep learning approach. We have investigated different Convolutional Neural Network (CNN) architectures and loss functions for optimization of the task at hand. Particular focus is given on in-network upsampling layers with learnable weights, aiming to optimally tackle the problem of high dimensional outputs without an excessive number of parameters. The best performing model encompasses residual learning and delivers state-of-the-art, real-time performance on depth prediction from images or videos of indoor and outdoor scenes. The methods developed within this project can be applicable to several other dense prediction problems as well.

Pictures

Figure 1: Results of depth estimation on benchmark datasets.
Figure 2: 3D reconstruction (point cloud) from a single image.

Team

Contact Person(s)

Christian Rupprecht
M.Sc. Christian Rupprecht
Iro Laina
M.Sc. Iro Laina

Working Group

Christian Rupprecht
M.Sc. Christian Rupprecht
Federico Tombari
Dr. Federico Tombari
Iro Laina
M.Sc. Iro Laina

Location



Technische Universität München
Institut für Informatik / I16
Boltzmannstr. 3
85748 Garching bei München

Tel.: +49 89 289-17058
Fax: +49 89 289-17059
Visit our lab at Garching.



internal project page

Please contact Iro Laina, Christian Rupprecht for available student projects within this research project.

Software

The trained models (indoors on NYU Depth v2; outdoors on Make3D) are publicly available and can be downloaded here:
Check out also our GitHub page, including the code for evaluation and inference.
The CNN-based depth estimation can be utilized by a monocular SLAM framework for successful scene reconstruction. For more information please visit this project .

Publications

2016
I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, N. Navab
Deeper Depth Prediction with Fully Convolutional Residual Networks (Oral Presentation)
International Conference on 3DVision (3DV), Stanford University, California, USA, October 2016.
The first two authors contribute equally to this paper.
(bib)
2017
K. Tateno , F. Tombari, I. Laina, N. Navab
CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, June, 2017.
The first two authors contribute equally to this paper.
(bib)


Edit | Attach | Refresh | Diffs | More | Revision r1.5 - 14 Jun 2017 - 10:24 - IroLaina

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif