Diploma thesis

Real-time Computation of Depth Maps with an
Application to Occlusion Handling in Augmented Reality.

Abstract

    An occasional problem encountered in state of the art Augmented Reality (AR) system is the problem of occlusions. Occlusion refers in this case to the problem of artificial images not being properly aligned in world space. To be more explicit they are missing a depth component and thus it may happen that artificial images are spuriously superimposed above real objects.

    This problem leads to the intend to find ways of detecting and handling occlusions. Most approaches in that direction are based on recovering depth information by means of stereo correspondence techniques. These techniques require an AR environment with two calibrated cameras mounted on a rugged stereo rig. In the AR domain this is usually a head mounted display, including two cameras, two displays (video see through, optical see through) and a tracking device.

    The main goal of this work was to evaluate different techniques for recovering depth maps. These techniques were evaluated in terms of accuracy, noise resistance, reproducibility and speed. Finally the task was to implement the most suitable one.

Milestones

Resources

• Template Numerical Toolkit (An Interface to scientific computing in C++)

Literature

All of the documents referenced occur in random order. They are thematically grouped and the groups occur in the order in which they are needed for the computation of the depth maps.

If anyone happens to find a mistake in the bibtex description please contact me.

1. Stereo Vision in general

3D-Szenenrekonstruktion aus Bilddaten. Einführung in den Stand der Technik @techreport{ stereo_basics,     author={Erich Rome and Marina Kolesnik},     title={{3D-S}zenenrekonstruktion aus Bilddaten. Einf{\"u}hrung in den Stand der Technik},     year=1998,     date={July 16, 1998},     address={Sankt Augustin, Germany},     institution={GMD},     number={MAKRO/VI/1},     url={http://ais.gmd.de/projects/Makro/reports/makro-report-VI-1.pdf} }

Kamerabasierte 3D-Rekonstruktion der Einsatzumgebung eines mobilen Roboters @misc{ v004_1999,     author = {M. Schmitt and J. Brodersen and G. Lietz and F. Lomberg and K.-F. Kraiss},     title = {Kamerabasierte 3D-Rekonstruktion der Einsatzumgebung eines mobilen Roboters},     year = 1999,     institution = {Lehrstuhl f{\"'u}r Technische Informatik, Rheinisch-Westf{\"a}lische Technische Hochschule Aachen (RWTH)},     url = {http://www.techinfo.rwth-aachen.de/Veroeffentlichungen/V004_1999.pdf} }

2. Image rectification for Stereo Vision

Computing Rectifying Homographies for Stereo Vision @inproceedings{ loopcomputing,     author = {C. Loop and Z. Zhang},     title = {Computing Rectifying Homographies for Stereo Vision},     pages = {125--131},     url = {http://citeseer.ist.psu.edu/loop99computing.html} }

A compact algorithm for rectification of stereo pairs @article{ fusiello00compact,     author = {Andrea Fusiello and Emanuele Trucco and Alessandro Verri},     title = {A compact algorithm for rectification of stereo pairs},     journal = {Machine Vision and Applications},     volume = {12},     number = {1},     pages = {16-22},     year = {2000},     url = "http://citeseer.ist.psu.edu/fusiello00compact.html} }

3. Disparity maps and the correspondence problem

Full search content independent block matching based on the Fast Fourier Transform @inproceedings{ fft-blockmatching,     author = {Steven L. Kilthau and Mark S. Drew and Torsten M{\"o}ller},     title = {Full search content independent block matching based on the Fast Fourier Transform},     booktitle = IEEE International Conference on Image Processing,     year = 2002,     url = {http://www.cs.sfu.ca/~mark/ftp/Icip02/index.htm} }

Dense Disparity Maps in Real-Time with an Application to Augmented Reality @misc{ schmidt-dense,     author = {Jochen Schmidt and Heinrich Niemann and Sebastian Vogt},     title = {Dense Disparity Maps in Real-Time with an Application to Augmented Reality},     url = {http://citeseer.ist.psu.edu/559569.html} }

Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation @inproceedings{ muehlmann-calculating,     author = {Karsten M{\"u}hlmann and Dennis Maier and J{\"u}rgen Hesser and Reinhard M{\"a}nner},     title = {Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation},     booktitle = IEEE Conference on Computer Vision and Pattern Recognition,     year = 2001,     month = dec,     address = {Kauai Marriott, Hawaii},     url = {http://citeseer.ist.psu.edu/560379.html} }

Predicting Disparity Windows for Real-Time Stereo @inproceedings{ mulligan00predicting,     author = {Jane Mulligan and Konstantinos Daniilidis},     title = {Predicting Disparity Windows for Real-Time Stereo},     booktitle = European Conference on Computer Vision,     pages = {220-235},     year = {2000},     url = {http://citeseer.ist.psu.edu/mulligan00predicting.html} }

A Fast Area-Based Stereo Matching Algorithm @misc{ vi2002,     author = {L. Di Stefano and M. Marchionni and S. Mattoccia and G. Neri},     title = {A Fast Area-Based Stereo Matching Algorithm},     url = {http://citeseer.ist.psu.edu/536377.html} }

A stereo machine for video-rate dense depth mapping and its new applications @inproceedings{ kanade96stereo,     author = {T. Kanade},     title = {A stereo machine for video-rate dense depth mapping and its new applications},     booktitle = IEEE Conference on Computer Vision and Pattern Recognition,     year = {1996},     url = {http://citeseer.ist.psu.edu/kanade96stereo.html} }