Self-supervising monocular 6D object pose estimation
a) Self6D[3] leverages synthetic data to train a base model for the task of 6D object pose estimation.
b) In addition, to close the domain gap between the synthetic and real world, they self-supervised the model on unlabeled RGB-D data enoforcing several constraints between predictions and data.
We offer a Master thesis project in collaboration with researchers from Google Zurich. We are looking for a motivated student interested in 3D computer vision and deep learning. The project involves in particular 6D object pose estimation [1,2], and self-supervised learning [3,4]. 6D object pose estimation describes the tasks of localizing an object of interest in an RGB image and subsequently estimating its 3D properties (i.e. 3D rotation and location). Example datasets and an online benchmark suite are hosted by [5].
While the field has recently made a lot of progress in accuracy and efficiency [5], many approaches rely on real annotated data. Nonetheless, obtaining annotated data for the task of pose estimation is often very time consuming and error prone. Moreover, when lacking appropriately labeled data the performance of these methods drops significantly [6]. Therefore, following recent trends in self-supervised learning, we want to investigate if we can train a deep model to learn purely from data without requiring any annotations, similar to [4] and [5].
Prerequisites: The candidate should have interest and knowledge in deep learning, be comfortable with Python and preferably have some experience with a deep learning framework, such as
PyTorch? or
TensorFlow?. Also, the candidate should have relevant prior experience with 3D computer vision, in terms of relevant university courses and/or projects.
Please drop an email to
PD Dr. Federico Tombari or
Fabian Manhardt to find out more.
[1] Wadim Kehl, Fabian Manhardt, Federico Tombari, Slobodan Ilic and Nassir Navab: “SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again”. ICCV 2017
[2] Mahdi Rad, Vincent Lepetit: “BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth”. ICCV 2017
[3] Gu Wang, Fabian Manhardt, Jianzhun Shao, Xiangyang Ji, Nassir Navab, Federico Tombari: “Self6D: Self-Supervised Monocular 6D Object Pose Estimation”. ECCV 2020
[4] Juil Sock, Guillermo Garcia-Hernando, Anil Armagan, Tae-Kyun Kim: “Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images”. 3DV 2020
[5] T. Hodaň et al.: “BOP: Benchmark for 6D Object Pose Estimation”. ECCV 2018
[6] Roman Kaskman, Sergey Zakharov, Ivan Shugurov, Slobodan Ilic: “HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects”. ICCVW 2019