MaPhotorealisticData

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine

3D Object Detection is essential for many tasks such a Robotic Manipulation or Augmented Reality. Nevertheless, recording appropriate real training data is difficult and time consuming. Due to this, many approaches rely on using synthetic data to train a Convolutional Neural Network [1, 2]. However, those approaches often suffer from overfitting to the synthetic world and do not generalize well to unseen real scenes. There are many works that try to address this problem [3, 4]. In this work we try to follow the work of [5], and intend to render photorealistic scenes in order to cope with this domain gap. Therefore, we will use a physics engine to generate physically plausible poses and use ray-tracing to render high-quality scenes.

References

[1] W. Kehl, F. Manhardt, S. Ilic, F. Tombari, and N. Navab. SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again. ICCV 2017
[2] M. Rad and V. Lepetit. Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth. ICCV 2017
[3] S. Hinterstoisser, V. Lepetit, P. Wohlhart, and K. Konolige. On pre-trained image features and synthetic images for deep learning. ECCV 2018
[4] S. Zakharov, W. Kehl, and S. Ilic. DeceptionNet?: Network-Driven Domain Randomization. Arxiv 2019
[5] T. Hodan, V. Vineet, R. Gal, E. Shalev, J. Hanzelka, T. Connell, P. Urbina, S. Sinha, and B. Guenter. Photorealisic image synthesis for object instance detection. Arxiv 2019

Students.ProjectForm
Title: Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine
Abstract: 3D Object Detection is essential for many tasks such a Robotic Manipulation or Augmented Reality. Nevertheless, recording appropriate real training data is difficult and time consuming. Due to this, many approaches rely on using synthetic data to train a Convolutional Neural Network. However, those approaches often suffer from overfitting to the synthetic world and do not generalize well to unseen real scenes. There are many works that try to address this problem. In this work we try to follow , and intend to render photorealistic scenes in order to cope with this domain gap. Therefore, we will use a physics engine to generate physically plausible poses and use ray-tracing to render high-quality scenes.
Student: Wessam Abdelbari Ali Frrag
Director: Federico Tombari
Supervisor: Fabian Manhardt, Johanna Wald
Type: Bachelor Thesis
Area: Computer Vision
Status: running
Start:  
Finish:  
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.4 - 06 Aug 2019 - 09:49 - FabianManhardt