Deep Learning-based 2D-3D Correspondence Matching for Object 6DoF Pose Estimation

Abstract

6DoF object pose estimation has a great meaning in augmented reality and robotic applications. More and more works tend to first find the 2D-3D correspondences between object model and image, and then solve the 6DoF pose using Perspective-n-Point (PnP^?) algorithm. We designed a new way to estimate the 2D-3D correspondences. The goal of the thesis is to implement the 2D-3D correspondence matching with an Auto Encoder-Decoder and compare the estimated object pose with state of the art approaches. The knowledge from Structure Light will also be used here to provide a robust matching.

Requirements

First experience in Deep Learning;

Familiar with one of the popular deep learning frameworks (PyTorch^?, MXNet, TensorFlow^?... );

Programming skills in C/C++, Python

Application

Please contact us with your latest transcript and CV for inquiries and applications.

Literature

Zakharov, Sergey, Ivan Shugurov, and Slobodan Ilic. "Dpod: 6d pose object detector and refiner." Proceedings of the IEEE International Conference on Computer Vision. 2019.

Park, Kiru, Timothy Patten, and Markus Vincze. "Pix2Pose: Pixel-wise coordinate regression of objects for 6D pose estimation." Proceedings of the IEEE International Conference on Computer Vision. 2019.