DaSurfaceForest

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

IDP / Dipl. / Master Thesis: Deformable Human Shape matching via Volumetric representation

Student: Bibiana do Canto Angonese
Advisor: Prof. Dr. Nassir Navab
Supervised by: Chun-Hao Paul Huang, Dr. Slobodan Ilic.

Overview

Recently there is a significant progress in human pose estimation with depth data. Thanks to decision forest, now people are able to infer correspondences (either sparse or dense) between a surface model and single depth image. Inspired by these works, in this project we would like to investigate the possibility to infer the correspondences between a surface model and a point cloud/visual hull obtained in multiple camera environments (right figure).

One expected issue is, what could be the proper "feature" of input? This feature has to more or less describe the location of the input point along the point cloud/visual hull, such that it's sufficiently discriminative to infer correspondences. 3D shape descriptors that have long been studied in 3D shape matching is therefore worthwhile a look.

This one-shot single frame pose estimation approach differs from standard ICP methods. It does not alternate between correspondence-estimation phase and energy-minimization phase. It is thus complement to normal human motion tracking approaches, which always rely on the result of previous frame to build correspondence pairs, and use it as the initialization of energy minimization. With the help of this one-shot approach, it is expected that one can prevent from error accumulation.
Inferring correspondences via regression forest

Requirements

  • The student should have experience in C++ and/or Matlab. Knowledge in random forest is also helpful.

Literature & Resource

Random forest for correspondence estimation: Keypoints and local descriptors of 3D shapes: Deformable surface tracking:

Publications

2017
C.-H. Huang, B. Allain, E. Boyer, J.-S. Franco, F. Tombari, N. Navab, S. Ilic
Tracking-by-Detection of 3D Human Shapes: from Surfaces to Volumes
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (bib)
2015
C.-H. Huang, F. Tombari, N. Navab
Repeatable Local Coordinate Frames for 3D Human Motion Tracking: from Rigid to Non-Rigid
International Conference on 3D Vision (3DV), Lyon, France, Oct. 20, 2015 (bib)
C.-H. Huang, E. Boyer, B. do Canto Angonese, N. Navab, S. Ilic
Toward User-specific Tracking by Detection of Human Shapes in Multi-Cameras
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, June 2015 (supp., poster) (bib)

Contact


Students.ProjectForm
Title: Deformable Human Shape matching via Volumetric representation
Abstract: Recently there is a significant progress in human pose estimation with depth data. Thanks to decision forest , now people are able to infer correspondences (either sparse or dense) between a surface model and single depth image. Inspired by these works, in this project we would like to investigate the possibility to infer the correspondences between a surface model and a point cloud/visual hull obtained from multiple camera environments.
Student: Bibiana do Canto Angonese
Director: Prof. Dr. Nassir Navab
Supervisor: Chun-Hao Paul Huang, Dr. Slobodan Ilic, Dr. Federico Tombari
Type: DA/MA/BA
Area: Computer Vision
Status: finished
Start: 01.02.2014
Finish: 31.10.2014
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.38 - 02 Sep 2016 - 07:50 - ChunHaoHuang