Bachelor Thesis: MRF as regularizations on forest predictions for matching meshes

Recently there has been a significant progress in human pose estimation from range maps and fully 3d data. Thanks to decision forest, now people can accurately infer correspondences (either sparse or dense) between a surface model and single depth image/visual hulls.

However, from time to time, the inferred correspondences do not respect the intrinsic structure of meshes, i.e., edge connectivity. In this project, we are interested in regularizing the associations between a surface model and a point cloud/visual hull obtained in multiple camera environments (right figure). Intuitively, vertices that lie in vicinity (in the geodesic sense) should attain predictions lying close to each other on the reference surface.

As a short term goal, we cast the problem as a graph optimization problem and apply CRF upon the forest predictions as post-processing step. As a long term goal, we would like to investigate how to merge CRF framework into forests, both in training and prediction. Namely, we hope that forests respect the mesh connectivity both during training and prediction. This goes into the direction of structured learning.