Research Issue: System Architectures for Ubiquitous Augmented Reality

Fachgebiet Augmented Reality (FAR)

General Description

In order to provide ubiquitous tracking, information presentation and interaction facilities, the underlying system architecture needs to be very flexible, adaptive and extensible. We have built an ad-hoc, peer-to-peer framework. We are evaluating and extending and/or simplifying the system. We are also exploring how to base the system on concepts and tools to specify, customize and analyze desired system behavior.

Associated People

• Permanent Staff: Martin Bauer, Florian Echtler, Manuel Huber, Gudrun Klinker, Daniel Pustka, Marcus Tönnis
• External Collaborators:
• Former Team Members: Thomas Reicher, Asa MacWilliams, Martin Wagner, Christian Sandor, Bernd Brügge

Associated Projects

trackframe

Use of AR technology in wide-range industry plants requires a tracking framework which supports a fusion of various tracking technologies based on careful evaluations of tradeoffs between technical requirements and costs. It is the goal of the trackframe project to systematically develop the formal basis of a tracking framework. The concept is expected to consider algorithmic and data-related standards via which tracking systems can be dynamically merged. Furthermore, trackframe aims at providing concepts towards the analysis of quality criteria (such as error statistics for individual sensors) and towards supporting AR-engineers in planning improvements to their tracker configuration. Three demonstrators will exhibit trackframe concepts in industrial and academic setups. (Bayerische Forschungsstiftung (BFS))

PRESENCCIA

This project undertakes a research programme that has as its major goal the delivery of presence in wide area distributed mixted reality environments. The environment will include a physical installation that people can visit both physicall and virtually. The installation will be the embodiment of an artificial intelligent entity that understands and learns from its interaction with people. People who inhabit the installation will at any one time be physically there, virtually there but remote, or entirely virtual beings with their own goals and capabilities for interacting with one another and with embodiments of real people. (EU 6th Framework, Future and Emerging Technologies (FET)): The Presenccia project

Ubiquitous Tracking

Augmented Reality applications are highly dependent from accurate and precise tracking data. Since current tracking technologies do not always provide such information everywhere in real-time application developer must combine certain trackers together to minimize the disadvantage of one tracker by another. These sensor networks can then be used to deliver positional relation information of objects to the application which then can be evaluated. By introducing the Ubitrack framework it is possible to form ubiquitous tracking environments which may consist of several sensor networks.

Demo Ubitrack

DWARF - Augmented Reality Framework

DWARF is a CORBA based framework that allows the rapid prototyping of distributed Augmented Reality applications. Augmented Reality is a new way for humans and computers to interact: real-world scenes are augmented with virtual objects. The augmented scene can, for example, be presented to the user through a see-through head-monted display. DWARF is a Framework for the development of such systems: a collection of intelligent modules that can be combined flexibly depending on the needs of the desired application.
The project started in July 2000.

Associated Publications

	M. Huber, M. Schlegel, G. Klinker Application of Time-Delay Estimation to Mixed Reality Multisensor Tracking Journal on Virtual Reality and Broadcasting (JVRB), Volume 11, 2014, online (22 pages), http://www.jvrb.org/past-issues/11.2014/3877 (bib)
	M. Tönnis, D. A. Plecher, G. Klinker Representing Information - Classifying the Augmented Reality Presentation Space Computers & Graphics, Vol. 37, Issue 8, Dec. 2013, pp 997-1011. (bib)
	D. Pustka, J. Hülß, J. Willneff, F. Pankratz, M. Huber, G. Klinker Optical Outside-In Tracking using Unmodified Mobile Phones The Eleventh IEEE and ACM International Symposium on Mixed and Augmented Reality, Atlanta, Georgia, Nov. 5 - 8, 2012, pp. 81-89. (bib)
	M. Huber, M. Schlegel, G. Klinker Temporal Calibration in Multisensor Tracking Setups 9. Workshop der GI-Fachgruppe Virtuelle und Erweiterte Realität (VR-AR 2012), Düsseldorf, Germany, Sept. 2012, pp. 201-212. (bib)
	D. Pustka, M. Huber, C. Waechter, F. Echtler, P. Keitler, J. Newman, D. Schmalstieg, G. Klinker Automatic Configuration of Pervasive Sensor Networks for Augmented Reality IEEE Pervasive Computing, Volume 10, Number 3, July-Sept. 2011, pp. 68-79. (bib)
	P. Keitler Management of Tracking and Tracking Accuracy in Industrial Augmented Reality Environments Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, April 2011 (online version) (bib)
	P. Keitler, D. Pustka, M. Huber, F. Echtler, G. Klinker Management of Tracking for Mixed and Augmented Reality Systems The Engineering of Mixed Reality Systems, E. Dubois, P. Gray, L. Nigay (eds.), Human-Computer Interaction Series, Springer Verlag, 2010. (bib)
	D. Pustka, J. Willneff, O. Wenisch, P. Lükewille, K. Achatz, P. Keitler, G. Klinker Determining the Point of Minimum Error for 3DOF Pose Uncertainty Representation The Ninth IEEE and ACM International Symposium on Mixed and Augmented Reality, Seoul, Korea, Oct. 13 - 16, 2010, pp. 37-45. (bib)
	P. Keitler, B. Becker, G. Klinker Management of Tracking for Industrial AR Setups The Ninth IEEE and ACM International Symposium on Mixed and Augmented Reality, Seoul, Korea, Oct. 13 - 16, 2010, pp. 73-82. (bib)
	P. Keitler, F. Pankratz, B. Schwerdtfeger, D. Pustka, , G. Klinker, C. Rauch, A. Chathoth, J. Collomosse , Y. Song Mobile Augmented Reality based 3D Snapshots Sechster Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR, Braunschweig, 18./19. November 2009 (bib)
	P. Keitler, F. Pankratz, B. Schwerdtfeger, D. Pustka, , G. Klinker, C. Rauch, A. Chathoth, J. Collomosse , Y. Song Mobile Augmented Reality based 3D Snapshots The 8th IEEE and ACM International Symposium on Mixed and Augmented Reality, Orlando, Florida, Oct. 19-22, 2009, pp. 199-200. (bib)
	M. Huber, M. Schlegel, G. Klinker Temporal Calibration in Multisensor Tracking Setups 8th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2009), Orlando, USA, October 2009, pp. 177-178 (bib)
	P. Keitler, M. Schlegel, G. Klinker Indirect Tracking to Reduce Occlusion Problems 4th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA, December 2008, pp. 224-235. (bib)
	D. Pustka, G. Klinker Dynamic Gyroscope Fusion in Ubiquitous Tracking Environments The Seventh IEEE and ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, Sep. 15 - 18, 2008, pp. 13-20. (bib)
	D. Pustka, M. Huber, G. Klinker Integrating Gyroscopes into Ubiquitous Tracking Environments IEEE Virtual Reality, Reno, Nevada, Mar. 8-12, 2008, pp. 283-284. (bib)
	M. Tönnis, J. Fischer, G. Klinker From Sensors to Assisted Driving - Bridging the Gap Journal of Software (JSW), Vol. 3, Issue 3, ISSN : 1796-217X, Mar. 2008, pp. 71-82. (bib)
	M. Huber, D. Pustka, P. Keitler, F. Echtler, G. Klinker A System Architecture for Ubiquitous Tracking Environments The Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 13 - 16, 2007, pp. 211-214. (bib)
	J. Newman, A. Bornik, D. Pustka, F. Echtler, M. Huber, D. Schmalstieg, G. Klinker Tracking for Distributed Mixed Reality Environments IEEE VR 2007 Workshop on 'Trends and Issues in Tracking for Virtual Environments', Charlotte, NC, USA, Mar 11, 2007 (bib)
	A. MacWilliams A Decentralized Adaptive Architecture for Ubiquitous Augmented Reality Systems Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, June 2005 (online version) (bib)
	M. Wagner Tracking With Multiple Sensors Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, Apr. 2005 (online version) (bib)
	C. Sandor, G. Klinker A Rapid Prototyping Software Infrastructure for User Interfaces in Ubiquitous Augmented Reality Journal for Personal and Ubiquitous Computing, Springer Verlag, 2005, pp. 169-185. The original publication is available online at www.springerlink.com . (bib)
	C. Sandor, B. Bell, A. Olwal, N. Temiyabutr, S. Feiner Visual End User Configuration of Hybrid User Interfaces Demo at International Symposium on Mixed and Augmented Reality (ISMAR) 2004, Arlington, VA, USA, Nov 2-4, 2004 (bib)
	J. Newman, M. Wagner, M. Bauer, A. MacWilliams, T. Pintaric, D. Beyer, D. Pustka, F. Strasser, D. Schmalstieg, G. Klinker Ubiquitous Tracking for Augmented Reality International Symposium on Mixed and Augmented Reality (ISMAR) 2004, Arlington, VA, USA, Nov 2-4, 2004, pp. 192-201. (bib)
	C. Sandor, B. Bell, A. Olwal, N. Temiyabutr, S. Feiner Visual End User Configuration of Hybrid User Interfaces Demo description in ACM SIGMM 2004 Workshop on Effective Telepresence, New York, NY, USA, Oct. 15, 2004. (bib)
	A. MacWilliams Software Development Challenges for Ubiquitous Augmented Reality First Virtual and Augmented Reality Workshop of the Gesellschaft für Informatik, Chemnitz, September 2004 (bib)
	M. Wagner Distributed Tracking with Multiple Sensors for Augmented Reality 1. Workshop "Virtuelle und Erweiterte Realität", TU Chemnitz, 27. - 28. September 2004 (bib)
	M. Wagner, A. MacWilliams, M. Bauer, G. Klinker, J. Newman, T. Pintaric, D. Schmalstieg Fundamentals of Ubiquitous Tracking Second International Conference on Pervasive Computing, Hot Spots section, Vienna, Austria, Apr. 18--23 2004 (bib)
	T. Reicher A Framework for Dynamically Adaptable Augmented Reality Systems Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, Apr. 2004 (online version) (bib)
	A. MacWilliams, T. Reicher, G. Klinker, B. Bruegge Design Patterns for Augmented Reality Systems Proc. of the International Workshop exploring the Design and Engineering of Mixed Reality Systems - MIXER 2004, Funchal, Madeira, CEUR Workshop Proceedings (bib)
	C. Kulas, C. Sandor, G. Klinker Towards a Development Methodology for Augmented Reality User Interfaces Proc. of the International Workshop exploring the Design and Engineering of Mixed Reality Systems - MIXER 2004, Funchal, Madeira, CEUR Workshop Proceedings (bib)
	O. Hilliges, C. Sandor, G. Klinker A Lightweight Approach for Experimenting with Tangible Interaction Metaphors Proc. of the International Workshop on Multi-user and Ubiquitous User Interfaces (MU3I) 2004, Funchal, Madeira (bib)
	J. Newman, M. Wagner, T. Pintaric, A. MacWilliams, M. Bauer, G. Klinker, D. Schmalstieg Fundamentals of Ubiquitous Tracking for Augmented Reality Technical Report TUM-I0323, Technische Universität München; Technical Report TR-188-2-2003-34, Vienna University of Technology (bib)
	M. Wagner, G. Klinker An Architecture for Distributed Spatial Configuration of Context Aware Applications 2nd International Conference on Mobile and Ubiquitous Multimedia, Norrköping, Sweden, Dec. 11-12 2003 (bib)
	A. MacWilliams, C. Sandor, M. Wagner, M. Bauer, G. Klinker, B. Bruegge Herding Sheep: Live System Development for Distributed Augmented Reality Proceedings of The Second International Symposium on Mixed and Augmented Reality (ISMAR 2003), pp. 123-132. (bib)
	T. Reicher, A. MacWilliams, B. Bruegge, G. Klinker Results of a Study on Software Architectures for Augmented Reality Systems Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR'03), 2003, pp. 274-275. (bib)
	M. Wagner, F. L{\"o}w Configuration Strategies of an AR Toolkit-based Wide Area Tracker The Second IEEE International Augmented Reality Toolkit Workshop, Tokyo, Japan, Oct. 7 2003. (bib)
	M. Bauer, O. Hilliges, A. MacWilliams, C. Sandor, M. Wagner, G. Klinker, J. Newman, G. Reitmayr, T. Pintaric, T. Fahmy, D. Schmalstieg Integrating Studierstube and DWARF International Workshop on Software Technology for Augmented Reality Systems. Tokyo, Japan, Oct. 7, 2003 (bib)
	M. Tönnis Data Management for Augmented Reality Applications Master Thesis - Technische Universitaet Muenchen (bib)
	M. Wagner Building Wide-Area Applications with the AR Toolkit The First IEEE International Augmented Reality Toolkit Workshop, Darmstadt, Germany, Sept. 29 2002. (bib)
	M. Bauer, B. Bruegge, G. Klinker, A. MacWilliams, T. Reicher, C. Sandor, M. Wagner An Architecture Concept for Ubiquitous Computing Aware Wearable Computers International Workshop on Smart Appliances and Wearable Computing. Vienna, Austria, July 2, 2002 (bib)
	G. Klinker, O. Creighton, A.H. Dutoit, R. Kobylinski, C. Vilsmeier, B. Bruegge Augmented maintenance of powerplants: A prototyping case study of a mobile AR system IEEE and ACM International Symposium on Aumgented Reality (ISAR 2001), New York, NY, October 29-30, 2001, pp. 124-136. (bib)
	M. Bauer, B. Bruegge, G. Klinker, A. MacWilliams, T. Reicher, S. Riß, C. Sandor, M. Wagner Design of a Component-Based Augmented Reality Framework Proceedings of The Second IEEE and ACM International Symposium on Augmented Reality (ISAR 2001), pp. 45-54. (bib)
	C. Sandor, T. Reicher CUIML: A Language for Generating Multimodal Human-Computer Interfaces Proceedings of the European UIML Conference, 2001 (bib)
	M. Wagner DWARF - Design, Prototypical Implementation and Testing of a Real-Time Optical Feature Tracker Diplomarbeit (bib)
	A. MacWilliams Using Ad-Hoc Services for Mobile Augmented Reality Systems Diplomarbeit (bib)
	G. Klinker, T. Reicher, B. Bruegge Distributed User Tracking Concepts for Augmented Reality Applications International Symposium on Aumgented Reality (ISAR 2000), Munich, Germany, 2000, pp. 37-44. (bib)
	G. Klinker, D. Stricker, D. Reiners Augmented Reality: A Balance Act between High Quality and Real-Time Constraints Proc. 1rst International Symposium on Mixed Reality (ISMR'99); published as a book: Y. Ohta and H. Tamura (eds.), Chapter 18 in "Mixed Reality - Merging Real and Virtual Worlds", 1999, pp. 325-346 (bib)
	G. Klinker, D. Stricker, D. Reiners The Use of Reality Models in Augmented Reality Applications Proc. European Workshop on 3D Structure from Multiple Images of Large-scale Environments (SMILE), in combination with ECCV'98, 1998; pp. 275-289. (bib)
	G. Klinker, K. Ahlers, D. Breen, P.-Y. Chevalier, C. Crampton, D. Greer, A. Kramer, E. Rose, M. Tuceryan, R. Whitaker Confluence of Computer Vision and Interactive Graphics for Augmented Reality PRESENCE - Teleoperators and Virtual Environments, Special Issue on Augmented Reality 6(4), 1997, pp. 433-451. (bib)