ProjectCNNSLAM

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

ProjectCNNSLAM

CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction

CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction

Scientific Director: Dr. Federico Tombari

Contact Person(s): Keisuke Tateno Dr. Federico Tombari

Keywords: Computer Vision

Abstract

Given the recent advances in depth prediction from Convolutional Neural Networks (CNNs), this paper investigates how predicted depth maps from a deep neural network can be deployed for accurate and dense monocular reconstruction. We propose a method where CNN-predicted dense depth maps are naturally fused together with depth measurements obtained from direct monocular SLAM. Our fusion scheme privileges depth prediction in image locations where monocular SLAM approaches tend to fail, e.g. along low-textured regions, and vice-versa. We demonstrate the use of depth prediction for estimating the absolute scale of the reconstruction, hence overcoming one of the major limitations of monocular SLAM. Finally, we propose a framework to efficiently fuse semantic labels, obtained from a single frame, with dense SLAM, yielding semantically coherent scene reconstruction from a single view. Evaluation results on two benchmark datasets show the robustness and accuracy of our approach.

Pictures

Figure 1: The proposed monocular SLAM approach (a) can estimate a much better absolute scale than the state of the art (b), which is necessary for many SLAM applications such as AR, \eg the skeleton is augmented into the reconstruction. c) our approach can yield joint 3D and semantic reconstruction from a single view.

Team

Contact Person(s)

Federico Tombari
Dr. Federico Tombari
Keisuke Tateno
Keisuke Tateno

Location



Technische Universität München
Institut für Informatik / I16
Boltzmannstr. 3
85748 Garching bei München

Tel.: +49 89 289-17058
Fax: +49 89 289-17059
Visit our lab at Garching.



internal project page

Please contact Keisuke Tateno Dr. Federico Tombari for available student projects within this research project.

Publications

2017
K. Tateno , F. Tombari, I. Laina, N. Navab
CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, June, 2017.
The first two authors contribute equally to this paper.
(bib)

Movie

CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction

Given the recent advances in depth prediction from Convolutional Neural Networks (CNNs), this paper investigates how predicted depth maps from a deep neural network can be deployed for accurate and dense monocular reconstruction. We propose a method where CNN-predicted dense depth maps are naturally fused together with depth measurements obtained from direct monocular SLAM. Our fusion scheme privileges depth prediction in image locations where monocular SLAM approaches tend to fail, e.g. along low-textured regions, and vice-versa. We demonstrate the use of depth prediction for estimating the absolute scale of the reconstruction, hence overcoming one of the major limitations of monocular SLAM. Finally, we propose a framework to efficiently fuse semantic labels, obtained from a single frame, with dense SLAM, yielding semantically coherent scene reconstruction from a single view. Evaluation results on two benchmark datasets show the robustness and accuracy of our approach.


Edit | Attach | Refresh | Diffs | More | Revision r1.4 - 15 Mar 2017 - 15:53 - KeisukeTateno

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif