MaEvaluationSelfSupervisedIndoorDepthEstimation

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

A study of existing self-supervised depth estimations

Abstract

Self-supervised depth estimation shows the promising result in the outdoor environment. However, there are few works target on the indoor or more arbitrary scenario.
The goal of this project is to investigate the current existing self-supervised methods on real-world scenarios, and possibly propose a way to improve there performance.

Requirements

Basic understanding of deep learning in computer vision
Basic Python and C++

Literature

--for some basic understanding--
Depth from Videos in the Wild:Unsupervised Monocular Depth Learning from Unknown Cameras
Deeper Depth Prediction with Fully Convolutional Residual Networks
Unsupervised Monocular Depth Estimation with Left-Right Consistency code
Unsupervised Learning of Depth and Ego-Motion from Video code
Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints code
Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction code

Results

Summary

This thesis studies unsupervised monocular depth prediction problem. Most of existing unsupervised depth prediction algorithms are developed under outdoor scenarios, while depth prediction in the indoor environment has long been ignored. Therefore this work focus on filling the gap by first evaluating existing architecture in the indoor environments and then improving the current design of architecture by solving observed issues in the first step.
After an extensive study and experiment in the current methods, an issue has been found that existing architecture cannot learn depth freely by the reason of the poor performance of the pose estimation network, which is a side network usually being trained together with depth prediction network. Unlike typical outdoor training sequence, such as Kitti dataset, indoor environment consist of more arbitrary camera movement and short baseline consecutive images which contribute poor training to the pose network. To address this issue, we propose two methods: First, we design a reconstruction loss function to provide extra constraint to the estimated pose and sharpen the predicted disparity map. Second, a novel neural network architecture is proposed to predict accurate 6-DOF pose. Our pose network combines the advantage of flowNet2 and poseNet which makes it able to learn to predict correct poses with relatively short baseline and arbitrary rotation training images. Apart from the above two methods, we use an ensemble and a flipping training techniques along with a median filter on the output disparity map, resulting outperformance of the current state-of-the-art unsupervised learning approaches.

Qualitative Result

groundTruth proposed video2depth
2213GT 2213Method_1 2213v2d

Quantitative Result

Method Datasets RMSE Abs Rel Sq Rel δ < 1.25 δ < 1.25^{2} δ < 1.25^{3}
struct2depth TUM-freiburg3 0.823 0.388 0.410 0.399 0.399 0.399
vid2depth TUM-freiburg3 1.067 0.308 0.915 0.710 0.710 0.710
vid2depth+ELWF TUM-freiburg3 0.910 0.275 0.649 0.711 0.711 0.711
vid2depth+ELWF+Median Filter TUM-freiburg3 0.877 0.268 0.604 0.715 0.715 0.715
Our Method TUM-freiburg3 0.920 0.285 0.656 0.697 0.697 0.697
Our Method +ELWF TUM-freiburg3 0.837 0.270 0.564 0.703 0.703 0.703
Our Method +ELWF+Median Filter TUM-freiburg3 0.812 0.264 0.527 0.707 0.707 0.707

Students.ProjectForm
Title: A study of existing self-supervised depth estimations
Abstract: Self-supervised depth estimation shows the promising result in the outdoor environment. However, there are few works target on the indoor or more arbitrary scenario.
The goal of this project is to investigate the current existing self-supervised methods on real-world scenarios and possibly propose a way to improve there performance.
Student: Yinglong Feng
Director: Federico Tombari
Supervisor: Shun-Cheng Wu
Type: DA/MA/BA
Area:  
Status: finished
Start: 1 April 2019
Finish: 30 Step 2019
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.4 - 06 Nov 2019 - 14:11 - ShunChengWu