3D Object Detection and Flow Estimation in Dynamic Surgical Environments

Abstract

3D object detection is the task of detecting 3D bounding box locations of objects from images or 3D environments. Whereas flow estimation is the extraction motion flow of the scene. In a temporal 3D environment, object detection and flow estimation is closely tied to each other, and can be used for reasoning from one another.
This IDP aims to connect these together, and establish a low-shot or self-supervised training procedure, which can be tremendously useful in the scenarios where large data collection is not easy, eg. in surgical environments.
One student will work on creating the 3D object detection pipeline from a state-of-the-art model, and one student will establish the 3D flow estimation. These two then be tied together by defining the relevant loss functions and dependencies.

Literature

Liu, Xingyu, Charles R. Qi, and Leonidas J. Guibas. "Flownet3d: Learning scene flow in 3d point clouds." CVPR 2019

Qi, Charles R., et al. "Deep hough voting for 3d object detection in point clouds." ICCV 2019.