Bachelor Thesis: MRF as regularizations on forest predictions for matching meshes
Student: Vera Hug
Advisor: Prof. Dr. Nassir Navab
Supervised by: Chun-Hao Paul Huang
, Dr. Federico Tombari
Recently there has been a significant progress in human pose estimation from range maps and fully 3d data. Thanks to decision forest, now people can accurately infer correspondences (either sparse or dense) between a surface model and single depth image/visual hulls.
However, from time to time, the inferred correspondences do not respect the intrinsic structure of meshes, i.e., edge connectivity. In this project, we are interested in regularizing the associations between a surface model and a point cloud/visual hull obtained in multiple camera environments (right figure). Intuitively, vertices that lie in vicinity (in the geodesic sense) should attain predictions lying close to each other on the reference surface.
As a short term goal, we cast the problem as a graph optimization problem and apply CRF upon the forest predictions as post-processing step. As a long term goal, we would like to investigate how to merge CRF framework into forests, both in training and prediction. Namely, we hope that forests respect the mesh connectivity both during training and prediction. This goes into the direction of structured learning.
- The student should have experience in C++ and/or Matlab. Knowledge in random forest is also helpful.
Literature & Resource
- E. Rodola, S. Bulo, T. Windheuser, M. Vestner and D. Cremers, Dense Non-Rigid Shape Correspondence using Random Forests (CVPR 2014)
- Q. Chen, and V. Koltun, Robust Nonrigid Registration by Convex Optimization (ICCV 2015)