3D Rigid Object Detection

Rigid 3D Object Detection Scientific Director: Federico Tombari, Slobodan Ilic Contact Person(s): Federico Tombari In industrial collaboration with: MVTec Software GmbH? Toyota Motor Corporation

Keywords: Computer Vision

Abstract

Object detection and localization is a crucial step for inspection and manipulation tasks in robotic and industrial applications. We present an object detection and localization scheme for 3D objects that combines intensity and depth data. A novel multimodal, scale- and rotation-invariant feature is used to simultaneously describe the object's silhouette and surface appearance. The object's position is determined by matching scene and model features via a Hough-like local voting scheme. The proposed method is quantitatively and qualitatively evaluated on a large number of real sequences, proving that it is generic and highly robust to occlusions and clutter. Comparisons with state of the art methods demonstrate comparable results and higher robustness with respect to occlusions.

Publications

2021
	P. Wang, F. Manhardt, L. Minciullo, L. Garattoni, S. Meier, N. Navab, B. Busam DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021 The first two authors contributed equally to this paper. (bib)
	Y. Di, F. Manhardt, G. Wang, X. Ji, F. Tombari, N. Navab SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation IEEE International Conference on Computer Vision (ICCV), October 2021 (bib)
	G. Wang, F. Manhardt, F. Tombari, X. Ji GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimationi IEEE Computer Vision and Pattern Recognition (CVPR), June 2021 (bib)
	L. Minciullo, F. Manhardt, K. Yoshikawa, S. Meier, F. Tombari, N. Kobori DB-GAN: Boosting Object Recognition Under Strong Lighting Conditions Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2021. The first two authors contributed equally to this paper. (bib)
2020
	F. Manhardt, G. Wang, B. Busam, M. Nickel, S. Meier, L. Minciullo, X. Ji, N. Navab CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning arXiv/2003.05848v3, September 2020 (bib)
	G. Wang, F. Manhardt, J. Shao, X. Ji, N. Navab, F. Tombari Self6D: Self-Supervised Monocular 6D Object Pose Estimation European Conference On Computer Vision (ECCV), Glasgow, Scotland, August [oral]. The first two authors contributed equally to this paper. (bib)
2019
	F. Manhardt, D. Arroyo, C. Rupprecht, B. Busam, T. Birdal, N. Navab, F. Tombari Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data International Conference on Computer Vision (ICCV), Seoul, Korea, October 2019 (bib)
	Y. Zhao, T. Birdal, H. Deng, F. Tombari 3D Point-Capsule Networks IEEE Computer Vision and Pattern Recognition (CVPR), Long Beach, United States, June 2019 (bib)
	T. Birdal, U. Simsekli Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope IEEE Computer Vision and Pattern Recognition (CVPR), Long Beach, United States, June 2019 [oral] (bib)
	F. Manhardt, W. Kehl, A. Gaidon ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape IEEE Computer Vision and Pattern Recognition (CVPR), Long Beach, United States, June 2019 The first two authors contributed equally. (bib)
	H. Deng, T. Birdal, S. Ilic 3D Local Features for Direct Pairwise Registration IEEE Computer Vision and Pattern Recognition (CVPR), Long Beach, United States, June 2019 (bib)
	Y. Gao, Y. Wang, , N. Navab, F. Tombari Variational Object-aware 3D Hand Pose from a Single RGB Image IEEE Robotics and Automation Letters 2019 (bib)
	T. Birdal, B. Busam, N. Navab, S. Ilic, P. Sturm Generic Primitive Detection in Point Clouds Using Novel Minimal Quadric Fits IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2019 (bib)
2018
	H. Deng, T. Birdal, S. Ilic PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors European Conference On Computer Vision (ECCV), Munich, Germany, September 2018 (bib)
	A. Haarbach, T. Birdal, S. Ilic Survey of Higher Order Rigid Body Motion Interpolation Methods for Keyframe Animation and Continuous-Time Trajectory Estimation International Conference on 3D Vision (3DV), Verona, Italy, September 2018 [spotlight] (bib)
	F. Manhardt, W. Kehl, N. Navab, F. Tombari Deep Model-Based 6D Pose Refinement in RGB European Conference On Computer Vision (ECCV), Munich, Germany, September 2018 [oral]. The first two authors contributed equally to this paper. (bib)
	T. Hodan, F. Michel, E. Brachmann, W. Kehl, A. Buch, D. Kraft, B. Drost, J. Vidal, S. Ihrke, X. Zabulis, C. Sahin, F. Manhardt, F. Tombari, T.K. Kim, J. Matas, C. Rother BOP: Benchmark for 6D Object Pose Estimation European Conference On Computer Vision (ECCV), Munich, Germany, September 2018. (bib)
	T. Birdal, B. Busam, N. Navab, S. Ilic, P. Sturm A Minimalist Approach to Type-Agnostic Detection of Quadrics in Point Clouds IEEE Computer Vision and Pattern Recognition (CVPR), Salt Lake City, United States, June 2018 (bib)
	H. Deng, T. Birdal, S. Ilic PPFNet: Global Context Aware Local Features for Robust 3D Point Matching IEEE Computer Vision and Pattern Recognition (CVPR), Salt Lake City, United States, June 2018 [spotlight] (bib)
2017
	W. Kehl, F. Tombari, S. Ilic, N. Navab Real-Time 3D Model Tracking in Color and Depth on a Single CPU Core Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, July 2017 (bib)
	T. Birdal, S. Ilic CAD Priors for Accurate and Flexible Instance Reconstruction International Conference on Computer Vision (ICCV), Venice, Italy, October 2017 (bib)
	W. Kehl, F. Manhardt, F. Tombari, S. Ilic, N. Navab SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again International Conference on Computer Vision (ICCV), Venice, Italy, October 2017 [oral]. The first two authors contributed equally to this paper. (bib)
	T. Birdal, S. Ilic A Point Sampling Algorithm for 3D Matching of Irregular Geometries IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, September 2017 (bib)
2016
	W. Kehl, F. Milletari, F. Tombari, S. Ilic, N. Navab Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation European Conference On Computer Vision (ECCV), Amsterdam, The Netherlands, October 2016 (bib)
2015
	W. Kehl, F. Tombari, N. Navab, S. Ilic, V. Lepetit Hashmod: A Hashing Method for Scalable 3D Object Detection British Machine Vision Conference (BMVC), Swansea, UK, September 2015 (bib)
	T. Birdal, S. Ilic Point Pair Features Based Object Detection and Pose Estimation Revisited IEEE Computer Society Conference on 3D Vision (3DV), Lyon, France, October 2015 [oral] (poster) (bib)
2013
	B. Drost, S. Ilic A Hierarchical Voxel Hash for Fast 3D Nearest Neighbor Lookup German Conference on Pattern Recognition (GCPR), Saarbruecken, Germany, September 2013. (bib)
2012
	S. Hinterstoisser, C. Cagniart, S. Ilic, P. Sturm, N. Navab, P. Fua, V. Lepetit Gradient Response Maps for Real-Time Detection of Texture-Less Objects IEEE Transactions on Pattern Analysis and Maschine Intelligence (TPAMI). (bib)
	S. Hinterstoisser, V. Lepetit, S. Ilic, S. Holzer, G. Bradski, K. Konolige, N. Navab Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes Asian Conference on Computer Vision (ACCV), Korea, Daejeon, November 2012 (bib)
	B. Drost, S. Ilic 3D Object Detection and Localization Using Multimodal Point Pair Features Second Joint 3DIM/3DPVT Conference: 3D Imaging, Modeling, Processing, Visualization & Transmission (3DIMPVT), Zurich, Switzerland, October 2012. (bib)
2011
	S. Hinterstoisser, S. Holzer, C. Cagniart, S. Ilic, K. Konolige, N. Navab, V. Lepetit Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes (Oral) IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, November 2011. (bib)
2010
	S. Hinterstoisser, V. Lepetit, S. Ilic, P. Fua, N. Navab Dominant Orientation Templates for Real-Time Detection of Texture-Less Objects IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	B. Drost, Markus Ulrich, N. Navab, S. Ilic Model Globally, Match Locally: Efficient and Robust 3D Object Recognition IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)

Team

Contact Person(s)

Federico Tombari

Working Group

Bertram Drost

Dr. David Tan

M.Sc. Fabian Manhardt

Federico Tombari

M.Sc. Keisuke Tateno

Dr. Slobodan Ilic

M.Sc. Tolga Birdal

M.Sc. Wadim Kehl

Alumni

Dr. Stefan Hinterstoisser

Location

Technische Universität München Institut für Informatik / I16 Boltzmannstr. 3 85748 Garching bei München Tel.: +49 89 289-17058 Fax: +49 89 289-17059

Visit our lab at Garching.



internal project page

Please contact Federico Tombari for available student projects within this research project.