ProjectInSeg

Real-time Incremental Segmentation on Dense SLAM Scientific Director: Dr. Federico Tombari Contact Person(s): Keisuke Tateno Dr. Federico Tombari

Keywords: Computer Vision

Abstract

This work proposes a real-time segmentation method for 3D point clouds obtained via Simultaneous Localization And Mapping (SLAM). The proposed method incrementally merges segments obtained from each input depth image in a unified global model using a SLAM framework. Differently from all other approaches, our method is able to yield segmentation of scenes reconstructed from multiple views in real-time, with a complexity that does not depend on the size of the global model.

Pictures

Figure 1: Real-time incremental segmentation applied to the reconstruction of a table-top scene.

Team

Contact Person(s)

Federico Tombari

M.Sc. Keisuke Tateno

Location

Technische Universität München Institut für Informatik / I16 Boltzmannstr. 3 85748 Garching bei München Tel.: +49 89 289-17058 Fax: +49 89 289-17059

Visit our lab at Garching.



internal project page

Please contact Keisuke Tateno Dr. Federico Tombari for available student projects within this research project.

Publications

2016
	K. Tateno, F. Tombari, N. Navab When 2.5D is not enough: Simultaneous Reconstruction, Segmentation and Recognition on dense SLAM IEEE International Conference on Robotics and Automation (ICRA), Stockholm, May 2016 (bib)

2015
	K. Tateno, F. Tombari, N. Navab Real-Time and Scalable Incremental Segmentation on Dense SLAM IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, September 2015 (bib)

Movie

	Real-Time and Scalable Incremental Segmentation on Dense SLAM This work proposes a real-time segmentation method for 3D point clouds obtained via Simultaneous Localization And Mapping (SLAM). The proposed method incrementally merges segments obtained from each input depth image in a unified global model using a SLAM framework. Differently from all other approaches, our method is able to yield segmentation of scenes reconstructed from multiple views in real-time, with a complexity that does not depend on the size of the global model.
	When 2.5D is not enough: Simultaneous Reconstruction, Segmentation and Recognition on dense SLAM While the main trend of 3D object recognition has been to infer object detection from single views of the scene --- i.e., 2.5D data --- this work explores the direction on performing object recognition on 3D data that is reconstructed from multiple viewpoints, under the conjecture that such data can improve the robustness of an object recognition system. To achieve this goal, we propose a framework which is able (i) to carry out incremental real-time segmentation of a 3D scene while being reconstructed via Simultaneous Localization And Mapping (SLAM), and (ii) to simultaneously and incrementally carry out 3D object recognition and pose estimation on the reconstructed and segmented 3D representations.
	Large Scale and Long Standing Simultaneous Reconstruction and Segmentation This work proposes a method to segment a 3D point cloud of a scene while simultaneously reconstructing it via Simultaneous Localization And Mapping (SLAM). The proposed method incrementally merges segments obtained from each input depth image in an unified global model leveraging the camera pose estimated via SLAM. Differently from other approaches, our method is able to yield segmentation of scenes reconstructed from multiple views in real-time and with a complexity that does not depend on the size of the global model. Moreover, we endow our system with two additional contributions: a loop closure approach and a failure recovery and re-localization approach, both specifically designed so to enforce global consistency between merged segments, thus making our system suitable for large scale and long standing reconstruction and segmentation.

Software

Precompiled library for real-time incremental segmentation on Dense SLAM, with dynamic linking openCV library on Windows, Linux, Mac - Version: Feb. 18th, 2016 InSeg.zip
Precompiled library for real-time incremental segmentation on Dense SLAM, with static linking openCV library on Windows - Version: Oct. 7th, 2015 InSeg_forStaticOpenCV.zip, This is suitable for precompiled openCV 3.0 library on Windows.