Hiwi project: Top-down 3D Human Pose/Shape Tracking without Background Subtraction

Student: -
Advisor: Prof. Dr. Nassir Navab
Supervised by: Chun-Hao Paul Huang, Dr. Slobodan Ilic
Due date:

Overview

In the past few years, top-down approaches have led to great successes in marker-less human motion capture (mocap). The term "top-down" means that a 3D reference surface (either generic or user specific) is pre-defined, and one has to track the motions (either surface deformations or skeletal poses) according to the observed 2D/3D data at each frame. The following videos demonstrate the examples of top-down approaches, using 3D visual hulls as input data:          Input visual hulls come from frame-wise 3D reconstructions using silhouettes from multiple cameras. A typical top-down tracking pipeline is illustrated in the right figure (top). Background subtraction therefore plays a crucial role, and consequently limits the applications to stay in indoor studios. In this project, we investigate the possibility to recover human motions directly with images, as shown in the right bottom image. The 3D surfaces are expected to be textured, or at least contain per-vertex color information. Thus, the problem amounts to maximizing the similarity between the projected texture of surfaces and the image observations.

Requirements

• The student can use whatever language he/she finds comfortable for fast prototyping, but the final solution is expected to be delivered in C++.
• Knowledge in optimization/energy minimizaition, and mesh processing is also helpful.

Literature & Resource

Blobby man series:

• C. Stoll, N. Hasler, J. Gall, H.-P. Seidel, and C. Theobalt, Fast Articulated Motion Tracking using a Sums of Gaussians Body Model (ICCV 2011)
  
• N. Robertini, E. de Aguiar, T. Helten, C. Theobalt, Efficient Multi-view Performance Capture of Fine-Scale Surface Detail (3DV 2014)
  
• S. Sridhar, H. Rhodin, H.-P. Seidel, A. Oulasvirta, C. Theobalt, Real-time Hand Tracking Using a Sum of Anisotropic Gaussians Model (3DV 2014)
  

Top-down human pose/shape tracking:

• C.-H. Huang, E. Boyer, and S. Ilic, Robust Human Body Shape and Pose Tracking (3DV 2013)
  
• C. Cagniart, E. Boyer, and S. Ilic, Probabilistic Deformable Surface Tracking From Multiple Videos (ECCV 2010)
  
• C. Cagniart, E. Boyer, and S. Ilic, Free-From Mesh Tracking : a Patch-Based Approach (CVPR 2010)
  
• Related project page: Spatio temporal modeling.
  

Contact

• Mail to Chun-Hao Paul Huang, or drop by room 03.13.039.