Photorealistic Rendering of Training Data for Object Detection and Pose Estimation with a Physics Engine

3D Object Detection is essential for many tasks such a Robotic Manipulation or Augmented Reality. Nevertheless, recording appropriate real training data is difficult and time consuming. Due to this, many approaches rely on using synthetic data to train a Convolutional Neural Network [1, 2]. However, those approaches often suffer from overfitting to the synthetic world and do not generalize well to unseen real scenes. There are many works that try to address this problem [3, 4]. In this work we try to follow the work of [5], and intend to render photorealistic scenes in order to cope with this domain gap. Therefore, we will use a physics engine to generate physically plausible poses and use ray-tracing to render high-quality scenes.

References

[1] W. Kehl, F. Manhardt, S. Ilic, F. Tombari, and N. Navab. SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again. ICCV 2017
[2] M. Rad and V. Lepetit. Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth. ICCV 2017
[3] S. Hinterstoisser, V. Lepetit, P. Wohlhart, and K. Konolige. On pre-trained image features and synthetic images for deep learning. ECCV 2018
[4] S. Zakharov, W. Kehl, and S. Ilic. DeceptionNet^?: Network-Driven Domain Randomization. Arxiv 2019
[5] T. Hodan, V. Vineet, R. Gal, E. Shalev, J. Hanzelka, T. Connell, P. Urbina, S. Sinha, and B. Guenter. Photorealisic image synthesis for object instance detection. Arxiv 2019