PhD Defense by Miroslava Slavcheva
|
Signed Distance Fields for Rigid and Deformable 3D Reconstruction
Abstract:Capturing three-dimensional environments is a key task in the growing fields of virtual and augmented reality. This thesis addresses the task of 3D reconstruction of both static and dynamic objects and scenes scanned with a single hand-held RGB-D camera, without any markers or prior knowledge.
Reconstructing rigid environments requires estimating the six degrees-of-freedom camera pose at every time instance, and subsequently fusing the acquired data into a geometrically consistent computer model. The task of reconstructing deformable objects is more challenging, as additionally the non-rigid motion that occurred in every frame has to be determined and factored out.
We propose to tackle both the rigid and deformable reconstruction problems via implicit-to-implicit alignment of SDF pairs without correspondence search. In the static case, we obtain more accurate pose estimates with a framework that permits straightforward incorporation of various additional constraints, such as surface colour and orientation. We start with the reconstruction of small- to medium-scale household objects and demonstrate how to extend the approach to larger spaces such as rooms. To this end, we develop a limited-extent volume strategy that restricts registration to the most geometrically distinctive regions of a scene, leading to significantly improved rotational motion estimation.
Finally, we adapt our approach to dynamic scenes by modifying our implicit-to-implicit approach so that new data is incremented appropriately. For this purpose we evolve an initial SDF to a target SDF by imposing rigidity constraints that require the underlying deformation field to be approximately Killing, i.e. volume-preserving and generating locally isometric motions. Alternatively, we employ gradient flow in the smooth Sobolev space, which favours global deformations over finer-scale details. These strategies also circumvent explicit correspondence search and thus avoid the repeated conversion between SDF and mesh representations that other techniques entail. Nevertheless, we ensure that correspondence information can be recovered by proposing two strategies based on Laplacian eigenfunctions, which are known to encode natural deformation patterns. Thanks to the used SDF representation, our non-rigid reconstruction approach is able to handle topological changes and fast motion, which are major obstacles for existing approaches.
| WebEventForm | |
|---|---|
| Title: | Signed Distance Fields for Rigid and Deformable 3D Reconstruction |
| Date: | 8 October 2018 |
| Location: | 01.06.020 |
| Abstract: | Capturing three-dimensional environments is a key task in the growing fields of virtual and augmented reality. This thesis addresses the task of 3D reconstruction of both static and dynamic objects and scenes scanned with a single hand-held RGB-D camera, without any markers or prior knowledge. Reconstructing rigid environments requires estimating the six degrees-of-freedom camera pose at every time instance, and subsequently fusing the acquired data into a geometrically consistent computer model. The task of reconstructing deformable objects is more challenging, as additionally the non-rigid motion that occurred in every frame has to be determined and factored out. We propose to tackle both the rigid and deformable reconstruction problems via implicit-to-implicit alignment of SDF pairs without correspondence search. In the static case, we obtain more accurate pose estimates with a framework that permits straightforward incorporation of various additional constraints, such as surface colour and orientation. We start with the reconstruction of small- to medium-scale household objects and demonstrate how to extend the approach to larger spaces such as rooms. To this end, we develop a limited-extent volume strategy that restricts registration to the most geometrically distinctive regions of a scene, leading to significantly improved rotational motion estimation. Finally, we adapt our approach to dynamic scenes by modifying our implicit-to-implicit approach so that new data is incremented appropriately. For this purpose we evolve an initial SDF to a target SDF by imposing rigidity constraints that require the underlying deformation field to be approximately Killing, i.e. volume-preserving and generating locally isometric motions. Alternatively, we employ gradient flow in the smooth Sobolev space, which favours global deformations over finer-scale details. These strategies also circumvent explicit correspondence search and thus avoid the repeated conversion between SDF and mesh representations that other techniques entail. Nevertheless, we ensure that correspondence information can be recovered by proposing two strategies based on Laplacian eigenfunctions, which are known to encode natural deformation patterns. Thanks to the used SDF representation, our non-rigid reconstruction approach is able to handle topological changes and fast motion, which are major obstacles for existing approaches. |
| Imageurl: |  |
| Type: | News |
| Videourl: | |
| Conferencelink: | |
