Vision Targeted CAD Models In academic collaboration with: Dr. Vincent Lepetit, EPFL, Lausanne Switzerland Scientific Director: Selim Benhimane and Nassir Navab Contact Person(s): Stefan Hinterstoisser

Keywords: Computer Vision, Industrial Augmented Reality

Abstract

This project investigates vision features and their intelligent application on known CAD models for pose estimation. The advantage of combining CAD models and vision features lies in the known 3D geometry of the CAD model. Exemplary vision features are e.g. SIFT features, Randomized Trees, SURF features and other well known methods. The main goal is to use such an extended CAD model for pose estimation and for tracking initialization.

Detailed Project Description

Natural 3D Markers (N3M): A first step towards vision features incorporating 3D-CAD models was made by creating Natural 3D Markers (N3Ms). N3Ms are minimal subsets of vision features which are learned during a training step from a 3D-CAD model and which are carefully designed to guarantee fast detection and quasi optimal pose estimation. N3Ms lead to high invariance to partial occlusion, high invariance to cluttered backgrounds, fast outlier elimination and the absence of ill-conditioned point configurations (e.g. collinear points) in the detection and pose estimation process. The results were published at ICCV 2007 in Rio de Janeiro.

The detection and pose estimations runs with about 15fps.

Videos

Tracking by Detection with Leopar

4 matched points are enough to detect their correct classification

Tracking (ESM) and N3Ms unified.

Publications

2012
	S. Hinterstoisser, C. Cagniart, S. Ilic, P. Sturm, N. Navab, P. Fua, V. Lepetit Gradient Response Maps for Real-Time Detection of Texture-Less Objects IEEE Transactions on Pattern Analysis and Maschine Intelligence (TPAMI). (bib)
2011
	J. Bohren, R. B. Rusu, E. G. Jones, E. Marder-Eppstein, C. Pantofaru, M. Wise, L. Mösenlechner, W. Meeussen, S. Holzer Towards Autonomous Robotic Butlers: Lessons Learned with the PR2 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011. (bib)
	S. Hinterstoisser, S. Holzer, C. Cagniart, S. Ilic, K. Konolige, N. Navab, V. Lepetit Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes (Oral) IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, November 2011. (bib)
	S. Hinterstoisser, V. Lepetit, S. Benhimane, P. Fua, N. Navab Learning Real-Time Perspective Patch Rectification International Journal of Computer Vision (IJCV), Springer Verlag, The final publication is available at www.springerlink.com (bib)
2010
	S. Holzer, S. Ilic, N. Navab Adaptive Linear Predictors for Real-Time Tracking IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	N. Alt, S. Hinterstoisser, N. Navab Rapid Selection of Reliable Templates for Visual Tracking IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	S. Hinterstoisser, V. Lepetit, S. Ilic, P. Fua, N. Navab Dominant Orientation Templates for Real-Time Detection of Texture-Less Objects IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
2009
	S. Hinterstoisser, O. Kutter, N. Navab, P. Fua, V. Lepetit Real-Time Learning of Accurate Patch Rectification (Oral Presentation) IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (USA), June 2009. (bib)
	S. Holzer, S. Hinterstoisser, S. Ilic, N. Navab Distance Transform Templates for Object Detection and Pose Estimation IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (USA), June 2009. (bib)
2008
	S. Hinterstoisser, S. Benhimane, V. Lepetit, P. Fua, N. Navab Simultaneous Recognition and Homography Extraction of Local Patches with a Simple Linear Classifier British Machine Vision Conference (BMVC), Leeds (UK), September 1-4, 2008 (bib)
	S. Hinterstoisser, S. Benhimane, N. Navab, P. Fua, V. Lepetit Online Learning of Patch Perspective Rectification for Efficient Object Detection IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (USA), June 2008. (bib)
2007
	S. Hinterstoisser, S. Benhimane, N. Navab N3M: Natural 3D Markers for Real-Time Object Detection and Pose Estimation IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, October 14-20, 2007 (bib)

Team

Contact Person(s)

Dr. Stefan Hinterstoisser

Working Group

Prof. Dr. Nassir Navab

Dr. Eng. Selim Benhimane

Location

Technische Universität München Institut für Informatik / I16 Boltzmannstr. 3 85748 Garching bei München Tel.: +49 89 289-17058 Fax: +49 89 289-17059

Visit our lab at Garching.



internal project page

Please contact Stefan Hinterstoisser for available student projects within this research project.