ProjectInSeg

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

ProjectInSeg

Real-time Incremental Segmentation on Dense SLAM

Real-time Incremental Segmentation on Dense SLAM

Scientific Director: Dr. Federico Tombari

Contact Person(s): Keisuke Tateno Dr. Federico Tombari

Keywords: Computer Vision

Abstract

This work proposes a real-time segmentation method for 3D point clouds obtained via Simultaneous Localization And Mapping (SLAM). The proposed method incrementally merges segments obtained from each input depth image in a unified global model using a SLAM framework. Differently from all other approaches, our method is able to yield segmentation of scenes reconstructed from multiple views in real-time, with a complexity that does not depend on the size of the global model.

Pictures

Figure 1: Real-time incremental segmentation applied to the reconstruction of a table-top scene.

Team

Contact Person(s)

Federico Tombari
Dr. Federico Tombari
Keisuke Tateno
Keisuke Tateno

Location



Technische Universität München
Institut für Informatik / I16
Boltzmannstr. 3
85748 Garching bei München

Tel.: +49 89 289-17058
Fax: +49 89 289-17059
Visit our lab at Garching.



internal project page

Please contact Keisuke Tateno Dr. Federico Tombari for available student projects within this research project.

Publications

2016
K. Tateno , F. Tombari, N. Navab
When 2.5D is not enough: Simultaneous Reconstruction, Segmentation and Recognition on dense SLAM
IEEE International Conference on Robotics and Automation (ICRA), Stockholm, May 2016 (bib)
2015
K. Tateno , F. Tombari, N. Navab
Real-Time and Scalable Incremental Segmentation on Dense SLAM
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, September 2015 (bib)

Movie

Real-Time and Scalable Incremental Segmentation on Dense SLAM

This work proposes a real-time segmentation method for 3D point clouds obtained via Simultaneous Localization And Mapping (SLAM). The proposed method incrementally merges segments obtained from each input depth image in a unified global model using a SLAM framework. Differently from all other approaches, our method is able to yield segmentation of scenes reconstructed from multiple views in real-time, with a complexity that does not depend on the size of the global model.

When 2.5D is not enough: Simultaneous Reconstruction, Segmentation and Recognition on dense SLAM

While the main trend of 3D object recognition has been to infer object detection from single views of the scene --- i.e., 2.5D data --- this work explores the direction on performing object recognition on 3D data that is reconstructed from multiple viewpoints, under the conjecture that such data can improve the robustness of an object recognition system. To achieve this goal, we propose a framework which is able (i) to carry out incremental real-time segmentation of a 3D scene while being reconstructed via Simultaneous Localization And Mapping (SLAM), and (ii) to simultaneously and incrementally carry out 3D object recognition and pose estimation on the reconstructed and segmented 3D representations.

Large Scale and Long Standing Simultaneous Reconstruction and Segmentation

This work proposes a method to segment a 3D point cloud of a scene while simultaneously reconstructing it via Simultaneous Localization And Mapping (SLAM). The proposed method incrementally merges segments obtained from each input depth image in an unified global model leveraging the camera pose estimated via SLAM. Differently from other approaches, our method is able to yield segmentation of scenes reconstructed from multiple views in real-time and with a complexity that does not depend on the size of the global model. Moreover, we endow our system with two additional contributions: a loop closure approach and a failure recovery and re-localization approach, both specifically designed so to enforce global consistency between merged segments, thus making our system suitable for large scale and long standing reconstruction and segmentation.

Software


Precompiled library for real-time incremental segmentation on Dense SLAM, with dynamic linking openCV library on Windows, Linux, Mac - Version: Feb. 18th, 2016 InSeg.zip
Precompiled library for real-time incremental segmentation on Dense SLAM, with static linking openCV library on Windows - Version: Oct. 7th, 2015 InSeg_forStaticOpenCV.zip, This is suitable for precompiled openCV 3.0 library on Windows.


Edit | Attach | Refresh | Diffs | More | Revision r1.13 - 15 Mar 2017 - 15:27 - KeisukeTateno

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif