Talk by Prof. Christian Theobalt

Speaker: Prof. Christian Theobalt, MPI Saarbrücken Date: Monday, 23 May 2011 Time: 13:00h Location: 03.13.010 (MI-Building, Campus Garching)

Title: Model-based Performance Capture from Video

Abstract: Performance capture is a term for a new class of acquisition methods that enable the simultaneous reconstruction of dynamic shape and surface appearance properties of dynamic scenes - in particular scenes involving human actors - from multi-view video. These reconstruction approaches exceed the capabilities of marker-based optical motion capture approaches that are still state-of-the-art in many real production scenarios, such as in movie productions. While marker-based systems can, at best, reconstruct skeletal motion or very sparse descriptions of non-rigid deformation, it is now possible to extract skeletal motion and full dynamic scene geometry, e.g., the shape of a person in general apparel, like a skirt, including wrinkles and folds on the surface. Even though performance capture algorithms deliver much richer data than traditional marker-based motion capture, many limitations remain and research is still at an early stage.

In our lab, we have been investigating model-based dynamic scene reconstruction methods that employ a template mesh to track a person in a scene. Recently, we tried to answer the question how model-based approaches can be extended to overcome some of the many limitations that still exist. For instance, we developed a new approach that enables us to capture skeletal motion and deforming surface geometry of several closely interacting people, which has been a challenge so far. We also developed a new approach to reconstruct fully-animatable models of humans from video. Since we detect cloth in the captured performances, and estimate paramaters of a physics-based cloth simulation model, we can change skeletal motion during postprocessing and obtain new animations of the entire character with plausible deformations. I will also briefly touch on some ideas we had about how to use performance capture in other application domains, such as model-based video editing and the synthesis of video-based characters, i.e., videos of a humans performing previously unseen user-designed motions.

Bio:

Christian Theobalt is the head of the research group "Graphics, Vision and Video" at the Max-Planck-Institut Informatik and a Professor of Computer Science at Saarland University, Saarbruecken, Germany. From 2007 until 2009 he was a Visiting Assistant Professor in the Department of Computer Science at Stanford University. He received his MSc degree in Artificial Intelligence from the University of Edinburgh, Scotland, and his Diplom (MS) degree in Computer Science from Saarland University, in 2000 and 2001 respectively. From 2001 to 2005 he was a researcher and PhD^? candidate in Hans-Peter Seidel's Computer Graphics Group at MPI Informatik. In 2005, he received his PhD^? (Dr.-Ing.) from Saarland University and MPI.

A lot of his research is about problems that lie on the boundary between the fields of Computer Vision and Computer Graphics, such as dynamic 3D scene reconstruction and marker-less motion capture, computer animation, appearance and reflectance modeling, machine learning for graphics and vision, new sensors for 3D acquisition, advanced video processing, as well as image- and physically-based rendering. For his work, he received several awards including the Otto Hahn Medal of the Max-Planck Society in 2007, and the EUROGRAPHICS Young Researcher Award in 2009.