Fabian Manhardt

Contact

Fabian Manhardt Email: fabian.manhardt[at]tum.de Position: PhD Candidate Address: Chair for Computer Aided Medical Procedures & Augmented Reality Fakultät für Informatik Technische Universität München Boltzmannstr. 3 85748 Garching b. München Office: MI 03.13.040

News

• Jul 27 (2021): 2 Papers accepted to ICCV'21
• Jun 30 (2021): 1 Paper accepted to IROS'21
• Mar 1 (2021): I'm finished!
• Mar 1 (2021): 1 Paper accepted to CVPR'21
• Nov 10 (2020): Our results for CPS++ on NOCS can be downloaded here
• Nov 6 (2020): Data for self-supervision from CPS++ can be downloaded here
• Nov 2 (2020): 1 Paper accepted to WACV '21
• Aug 12 (2020): The code for differential rendering from Self6D can be found here
• Jul 2 (2020): 1 Paper accepted to ECCV '20 (Oral, ~2% acceptance)
• Jul 23 (2019): 1 Paper accepted to ICCV '19
• Feb 25 (2019): 1 Paper accepted to CVPR '19
• Jan 9 (2019): Inference Code of the ECCV '18 paper can be found here
• Dec 17 (2018): My talk at ECCV '18 can be found here
• Jul 3 (2018): 2 Papers accepted to ECCV '18 (1 first author as oral, ~2% acceptance).
• Oct 31 (2017): Inference Code of the ICCV '17 paper can be found here
• Aug 5 (2017): Our SSD6D paper at ICCV '17 in Venice just got selected as Oral (~2% acceptance) presentation

Research Topics

I am currently involved in conducting research in the following research areas.

• Computer Vision.
• Deep Learning.

Student Projects

If you are interested in projects in the area of Computer Vision and Deep Learning, you are always welcome to drop by my office.

Available
DA/MA/BA	Self-supervising monocular 6D object pose estimation (Fabian Manhardt, Federico Tombari)
Master Thesis	3D Scene Understanding Leveraging Scene Graphs (Helisa Dhamo Fabian Manhardt, Federico Tombari)

Running

Finished
Bachelor Thesis	Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine (Fabian Manhardt, Johanna Wald, Federico Tombari)
Master Thesis	Incremental Learning For Robotic Grasping (Fabian Manhardt, Nassir Navab)
Bachelor Thesis	Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine (Fabian Manhardt, Johanna Wald, Federico Tombari)
Master Thesis	Dealing with the ambiguity induced by object symmetry in pose estimation tasks (Fabian Manhardt, Federico Tombari, Prof. Nassir Navab)
Master Thesis	Class-Level Object Detection and Pose Estimation from a Single RGB Image Only (Fabian Manhardt, Federico Tombari, Prof. Nassir Navab)

Teaching Assistance

• Hauptseminar Computer Vision and Deep Learning for Autonomous Driving (2020WiSe)
• Praktikum Perception and Learning in Robotics and Augmented Reality (2020SoSe)
• Hauptseminar Recent Trends in 3D Computer Vision and Deep Learning (2019WiSe)
• Hauptseminar Computer Vision and Deep Learning for Autonomous Driving (2019SoSe)
• Hauptseminar Computer Vision and Deep Learning for Autonomous Driving (2018WiSe)
• Hauptseminar Foundations of Computer Vision (2018WiSe)
• Praktikum Perception and Learning in Robotics and Augmented Reality (2018SoSe)
• Hauptseminar Foundations of Computer Vision (2017WiSe)
  
  
• Hauptseminar Recent Trends in 3D Computer Vision and Deep Learning (Tutor) (2018WiSe)
• Hauptseminar Recent Trends in 3D Computer Vision and Deep Learning (Tutor) (2017WiSe)
• Praktikum Perception and Learning in Robotics and Augmented Reality (Tutor) (2017SoSe)

List of Publications

2021
	P. Wang, F. Manhardt, L. Minciullo, L. Garattoni, S. Meier, N. Navab, B. Busam DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021 The first two authors contributed equally to this paper. (bib)
	H. Dhamo, F. Manhardt, N. Navab, F. Tombari Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes using Scene Graphs IEEE International Conference on Computer Vision (ICCV), October 2021 The first two authors contributed equally. (bib)
	Y. Di, F. Manhardt, G. Wang, X. Ji, F. Tombari, N. Navab SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation IEEE International Conference on Computer Vision (ICCV), October 2021 (bib)
	G. Wang, F. Manhardt, F. Tombari, X. Ji GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimationi IEEE Computer Vision and Pattern Recognition (CVPR), June 2021 (bib)
	L. Minciullo, F. Manhardt, K. Yoshikawa, S. Meier, F. Tombari, N. Kobori DB-GAN: Boosting Object Recognition Under Strong Lighting Conditions Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2021. The first two authors contributed equally to this paper. (bib)
2020
	F. Manhardt, G. Wang, B. Busam, M. Nickel, S. Meier, L. Minciullo, X. Ji, N. Navab CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning arXiv/2003.05848v3, September 2020 (bib)
	G. Wang, F. Manhardt, J. Shao, X. Ji, N. Navab, F. Tombari Self6D: Self-Supervised Monocular 6D Object Pose Estimation European Conference On Computer Vision (ECCV), Glasgow, Scotland, August [oral]. The first two authors contributed equally to this paper. (bib)
2019
	F. Manhardt, D. Arroyo, C. Rupprecht, B. Busam, T. Birdal, N. Navab, F. Tombari Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data International Conference on Computer Vision (ICCV), Seoul, Korea, October 2019 (bib)
	F. Manhardt, W. Kehl, A. Gaidon ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape IEEE Computer Vision and Pattern Recognition (CVPR), Long Beach, United States, June 2019 The first two authors contributed equally. (bib)
2018
	F. Manhardt, W. Kehl, N. Navab, F. Tombari Deep Model-Based 6D Pose Refinement in RGB European Conference On Computer Vision (ECCV), Munich, Germany, September 2018 [oral]. The first two authors contributed equally to this paper. (bib)
	T. Hodan, F. Michel, E. Brachmann, W. Kehl, A. Buch, D. Kraft, B. Drost, J. Vidal, S. Ihrke, X. Zabulis, C. Sahin, F. Manhardt, F. Tombari, T.K. Kim, J. Matas, C. Rother BOP: Benchmark for 6D Object Pose Estimation European Conference On Computer Vision (ECCV), Munich, Germany, September 2018. (bib)
2017
	W. Kehl, F. Manhardt, F. Tombari, S. Ilic, N. Navab SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again International Conference on Computer Vision (ICCV), Venice, Italy, October 2017 [oral]. The first two authors contributed equally to this paper. (bib)
2015
	A. Luckow, K. Kennedy, F. Manhardt, E. Djerekarov, A. Apon Automotive big data: Applications, workloads and infrastructures International Conference on Big Data (Big Data), Santa Clara, CA, USA, December 2015 (bib)

SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again

Deep Model-Based 6D Pose Refinement in RGB

We present a novel approach for model-based 6D pose refinement in color data. Building on the established idea of contour-based pose tracking, we teach a deep neural network to predict a translational and rotational update. At the core, we propose a new visual loss that drives the pose update by aligning object contours, thus avoiding the definition of any explicit appearance model.

ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape

We present a deep learning method for end-to-end monocular 3D object detection and metric shape retrieval. We propose a novel loss formulation by lifting 2D detection, orientation, and scale estimation into 3D space. Instead of optimizing these quantities separately, the 3D instantiation allows to properly measure the metric misalignment of boxes.

Explaining the Ambiguity of Object Detection and 6D Pose from Visual Data

In this work we propose to explicitly deal with visual ambiguities in 6D pose estimation tasks. Therefore, for each object instance we predict multiple 6D pose outcomes to estimate the specific pose distribution generated by symmetries and repetitive textures.

CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning

We propose a novel method for class-level monocular 6D pose estimation, coupled with metric shape retrieval. We leverage a new loss formulation which directly optimizes over all parameters, i.e. 3D orientation, translation, scale and shape at the same time. Instead of decoupling each parameter, we transform the regressed shape, in the form of a point cloud, to 3D and directly measure its metric misalignment. Unfortunately, acquiring adequate annotations is very time-consuming and labor intensive. To overcome this shortcoming, we additionally propose the idea of synthetic-to-real domain transfer for class-level 6D poses by means of self-supervised learning, which removes the burden of collecting numerous manual annotations. In essence, after training our proposed method fully supervised with synthetic data, we leverage recent advances in differentiable rendering to self-supervise the model with unannotated real RGB-D data to improve latter inference. Data can be downloaded here. Results for CPS++ on NOCS can be downloaded here