MaPhotorealisticDataExtension

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine

3D Object Detection is essential for many tasks such a Robotic Manipulation or Augmented Reality. Nevertheless, recording appropriate real training data is difficult and time consuming. Due to this, many approaches rely on using synthetic data to train a Convolutional Neural Network [1, 2]. However, those approaches often suffer from overfitting to the synthetic world and do not generalize well to unseen real scenes. There are many works that try to address this problem [3, 4]. In this work we try to follow the work of [5], and intend to render photorealistic scenes in order to cope with this domain gap. Therefore, we will use a physics engine to generate physically plausible poses and use ray-tracing to render high-quality scenes. In this particular work, we will extend another thesis to improve the renderings' quality as e.g. enhance the renderings' realism in terms of lightning and reflection.

References

[1] W. Kehl, F. Manhardt, S. Ilic, F. Tombari, and N. Navab. SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again. ICCV 2017
[2] M. Rad and V. Lepetit. Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth. ICCV 2017
[3] S. Hinterstoisser, V. Lepetit, P. Wohlhart, and K. Konolige. On pre-trained image features and synthetic images for deep learning. ECCV 2018
[4] S. Zakharov, W. Kehl, and S. Ilic. DeceptionNet?: Network-Driven Domain Randomization. ICCV 2019
[5] T. Hodan, V. Vineet, R. Gal, E. Shalev, J. Hanzelka, T. Connell, P. Urbina, S. Sinha, and B. Guenter. Photorealisic image synthesis for object instance detection. ICIP 2019

Students.ProjectForm
Title: Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine
Abstract: 3D Object Detection is essential for many tasks such a Robotic Manipulation or Augmented Reality. Nevertheless, recording appropriate real training data is difficult and time consuming. Due to this, many approaches rely on using synthetic data to train a Convolutional Neural Network. However, those approaches often suffer from overfitting to the synthetic world and do not generalize well to unseen real scenes. There are many works that try to address this problem. In this work we try to follow , and intend to render photorealistic scenes in order to cope with this domain gap. Therefore, we will use a physics engine to generate physically plausible poses and use ray-tracing to render high-quality scenes. In this particular work, we will extend another thesis to improve the renderings' quality as e.g. enhance the rendering realism in terms of lightning and reflection.
Student:  
Director: Federico Tombari
Supervisor: Fabian Manhardt, Johanna Wald
Type: DA/MA/BA
Area:  
Status: open
Start: ASAP
Finish:  
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.2 - 06 Aug 2019 - 09:06 - FabianManhardt