[27.05.15] Our paper "A Bayesian Approach to Multi-view 4D Modeling" got accepted for International Journal of Computer Vision (IJCV).
[02.03.15] Our paper "Toward User-specific Tracking by Detection of Human Shapes in Multi-Cameras" got accepted as a poster presentation at CVPR 2015.
[01.02.15] I'm visiting Morpheo team in INRIA (Grenoble, France) from 1st of February til 13th of February.
[30.06.14] CVPR`14 sequences released! Besides images, silhouettes and reconstructed visual hulls, we also provide reference surfaces, rigged skeletons, and annotated joint locations (in 5 cameras). Check the following links: WalkChair, and HammerTable & SideSit.
[01.03.14] Our paper "Human Shape and Pose Tracking Using Keyframes" got accepted as a poster presentation at CVPR 2014
[18.02.14] Previous work at Academia Sinica, "Heterogeneous Domain Adaptation and Classification by Exploiting the Correlation Subspace" got accepted for TIP (demo code).
[30.06.13] Our 3DV oral paper "Robust Human Body Shape and Pose Tracking" was awarded as best paper runner up ! (link)
A great part of Computer Vision research has been dedicated to shape recovery, tracking and detection of 3D objects in images and videos. While excellent results have been achieved in these areas, the majority of the methods still assume static scenes and rigid objects, and rarely explore temporal information. However, the world surrounding us is highly dynamic, and in many situations objects deform over time. This temporal information provides a richer and denser source of information and have not yet been extensively exploited. Our objective within this project is to explore spatio-temporal information in order to recover 3D shapes and the motion of the deformable objects. Therefore, we refer to this area as spatio-temporal or four dimensional modeling (4D modeling). With the increased popularity of 3D content in film industry, TV, Internet and games, tools and methods that exploit spatio-temporal information and allow fast and automated 3D content production are going to be indispensable.
Student Projects for Hiwi and Dipl./B.Sc./M.Sc. Thesis
Inspired by the recent work that applies random forests to estimate human poses with depth data (either via body part classification or dense correspondences), in this project we investigate the possibility to infer the dense correspondences between a surface model and a complete 3D visual hull obtained in multiple camera environments. With frame-wise data-model associations, our method is able to detect human shapes in a clutter scene, which we term tracking-by-detection of 3D human shapes.
The inferred data-model associations from the forests often look noisy because it does not respect the intrinsic structure, i.e. triangles/edge connectivity. In this project, we mitigate this by proposing a new way to aggregate leaf-node predictions from forests. As the correspondences are highly interdependent among the local neighborhood, we apply random fields (MRF/CRF) to regularize them, yield the optimal configuration w.r.t. the whole shape. The aggregation is therefore mesh-based rather than vertex-based.
Volumetric Tracking by Detection
Following the above tracking-by-detection concept, in this project we investigate different parameterizations of 3D shapes. We choose centroidal Voronoi tessellation (CVT) as the shape representation and realize discriminative correspondences in a volumetric manner. We further propose a unified volumetric tracking-by-detection pipeline, where the shape representation, deformation model, feature description, and primitive association are all built on a single consistent representation, CVT.
In this project, we investigate the possibility to recover human motions directly with images, without doing background subtraction. The 3D surfaces are expected to be textured, or at least contain per-vertex color information. Thus, the problem amounts to maximizing the similarity between the projected texture of surfaces and the image observations.
Y.-R. Yeh, C.-H. Huang, Y.-C. F. Wang, Heterogeneous Domain Adaptation and Classification by Exploiting the Correlation Subspace, IEEE Transactions on Image Processing (TIP), vol. 23, no. 5, pp. 2009-2018, May. 2014 (pdf)
C.-H. Huang, Y.-R. Yeh, Y.-C. F. Wang, Recognizing Unseen Actions across Cameras by Exploring the Correlated Subspace, European Conference on Computer Vision (ECCV), The 4th International VECTaR? Workshop, Firenze, Italy, October 12, 2012 (pdf, slides)
H.-M. Wang, C.-H. Huang, J.-F. Yang, Block-Based Depth Maps Interpolation for Efficient Multiview Content Generation, IEEE Transactions On Circuits And Systems for Video Technology (TCSVT), vol. 21, no. 12, pp. 1847-1858, Dec. 2011
H.-M. Wang, C.-H. Huang, J.-F. Yang, Depth Maps Interpolation from Existing Pairs of Keyframes and Depth Maps for 3D Video Generation, IEEE International Symposium on Circuits and Systems (ISCAS), Paris, France, May 30 - June 2, 2010