Fast Reconstruction of the World from Photos and Videos
Jan-Michael Frahm
Abstract
In recent years photo/video sharing web sites like Flickr and
YouTube? have become increasingly
popular. Nowadays, every day terra bytes of photos and videos are uploaded. These data survey
large parts of the world throughout the di fferent seasons, various weather conditions and all times
of the day. In the talk I will present my work on the highly efficient reconstruction of 3D models
from these data. It addresses a variety of the current challenges that have to be addressed to
achieve a concurrent 3D model from these data. The challenges are: estimation of the geometric
and radiometric camera calibration from videos and photos, efficient robust camera motion estima-
tion for (quasi-)degenerate estimation problems, high performance stereo estimation from multiple
views, automatic selection of correct views from noisy image/video collections, image based lo-
cation recognition for topology detection. In the talk I will discuss the details of our real-time
camera motion estimation from video using our Adaptive Real-Time Random Sample Consensus
(ARRSAC) and our high performance salient feature tracker, which simultaneously estimates the
radiometric camera calibration and tracks the motion of the salient feature points. Furthermore
our technique to achieve robustness against (quasi-) degenerate data will be introduced. It allows
to detect and overcome the case of data, which under-constrain the camera motion estimation
problem. Additionally our optimal stereo technique for determining the scene depths with con-
stant precision throughout the scene volume will be explained during the talk. It allows to perform
the scene depth estimation from a large set of views with optimal computational e ffort while ob-
taining the depth with constant precision throughout the reconstruction volume. I also discuss our
fast technique for the image based location recognition, which uses commodity graphics processors
to achieve real-time performance while providing high recognition rates. Furthermore in the talk
I present our work on 3D reconstruction from internet photo collections. It combines image based
recognition with geometric constraints to efficiently perform the simultaneous selection of correct
views and the 3D reconstruction from large collections of photos. The talk will also explain the
future challenges in all the mentioned areas.
Short Biography
Jan-Michael Frahm is a Research Assistant Professor at University of North Carolina at Chapel
Hill. He received his Ph.D in computer vision in 2005 from the Christian-Albrechts University
of Kiel, Germany. His dissertation, "Camera Self-Calibration with Known Camera Orientation"
received the prize for that years best Ph.D. dissertation in CAUs College of Engineering. His
Diploma in Computer Science is from the University of Lbeck. Dr.-Ing. Frahms research interests
include a variety of computer vision problems. He has worked on structure from motion for
single/multi-camera systems for static and dynamic scenes to create 3D models of the scene; real-
time multi-view stereo to create a dense scene geometry from camera images; use of camera-sensor
systems for 3D scene reconstruction with fusion of multiple orthogonal sensors; improved robust
and fast estimation methods from noisy data to compensate for highly noisy measurements in
various stages of the reconstruction process; high performance feature tracking for salient image-
point motion extraction; and the development of data-parallel algorithms for commodity graphics
hardware for efficient 3D reconstruction.
