Research Issue: Ubiquitous Tracking - UbiTrack

Fachgebiet Augmented Reality (FAR)

General Description

Augmented Reality provides a natural interface to the calm pervasive technology anticipated in large-scale Ubiquitous Computing environments. However, the range of classic AR applications has been limited by the scope, range and cost of sensors used for tracking. Hybrid tracking approaches can go some way to extending this range. In our approach, called Ubiquitous Tracking (Ubitrack), data from widespread and diverse heterogeneous tracking sensors is automatically and dynamically fused, and then transparently provided to applications.

We are currently investigating the following aspects of Ubiquitous Tracking:

Formal System Model In order to formally describe tracking setups, we introduced the Spatial Relationship Graph (SRG). In this graph, nodes correspond to objects and coordinate frames, while edges represent tracked or otherwise known transformations, attributed with quality-of-service information. Using this model, clients can query particular spatial relationships and give the minimal tracking quality that is required.

Spatial Relationship Patterns All algorithms used for tracking and calibration can be mapped to particular patterns in the Spatial Relationship Graph. By looking for such patterns in an SRG, the Ubitrack middleware can, given the description of a tracking setup, create data flow networks to fulfill a client's request.

System Architecture For building real Ubitrack systems, a software system is necessary to register sensors, client queries and to construct data flow networks at runtime that perform the necessary computations on the raw sensor measurements in order to fullfill a client's needs. For truely ubiquitous applications, our Ubitrack implementation is based on the DWARF middleware which allows for dynamic service discovery in a peer-to-peer fashion. In industrial applications, we take the more pragmatic approach of using a centralized Ubitrack server. This functionality will be provided in an open-source Ubitrack library.

Error Analysis As Ubitrack systems have to combine measurements of previously unknown sensors, accurate information on the characteristics of sensors, measurements and transformations needs to be available. For that, we perform error analysis of sensors and algorithms used for tracking and calibration.

Associated People

• Permanent Staff: Florian Echtler, Manuel Huber, Gudrun Klinker, Daniel Pustka
• External Collaborators: Benjamin Becker (EADS), Michael Schlegel (BMW), Peter Keitler (peyclon)
• Former Team Members: Martin Bauer, Asa MacWilliams, Martin Wagner

Associated Projects

DySenNetz (Dynamic Sensor Networks)

This project provides the foundations and formal basis towards building dynamic systems to fuse multiple sensors online. To this end, it a) lays the groundwork for building ad-hoc tracker networks for Augmented Reality, and b) formulates and analyzes accumulation of measurement errors and sensor noise. (Deutsche Forschungsgesellschaft (DFG))

PRESENCCIA

This project undertakes a research programme that has as its major goal the delivery of presence in wide area distributed mixted reality environments. The environment will include a physical installation that people can visit both physicall and virtually. The installation will be the embodiment of an artificial intelligent entity that understands and learns from its interaction with people. People who inhabit the installation will at any one time be physically there, virtually there but remote, or entirely virtual beings with their own goals and capabilities for interacting with one another and with embodiments of real people. (EU 6th Framework, Future and Emerging Technologies (FET)): The Presenccia project

trackframe

Use of AR technology in wide-range industry plants requires a tracking framework which supports a fusion of various tracking technologies based on careful evaluations of tradeoffs between technical requirements and costs. It is the goal of the trackframe project to systematically develop the formal basis of a tracking framework. The concept is expected to consider algorithmic and data-related standards via which tracking systems can be dynamically merged. Furthermore, trackframe aims at providing concepts towards the analysis of quality criteria (such as error statistics for individual sensors) and towards supporting AR-engineers in planning improvements to their tracker configuration. Three demonstrators will exhibit trackframe concepts in industrial and academic setups. (Bayerische Forschungsstiftung (BFS))

Spatial Relationship Pattern

A spatial relationship pattern denotes a subproblem in a complex tracking or calibration setup graph for which one or more known algorithms exist. The pattern defines the ‘signature’ of an algorithms to solve the specific problem in a general setup.

The project aims at identifying, describing and analyzing various spatial relationship patterns.

Associated Publications

	U. Eck, F. Pankratz, C. Sandor, G. Klinker, H. Laga Precise Haptic Device Co-Location for Visuo-Haptic Augmented Reality IEEE Transactions on Computer Graphics and Visualization, vol. 21, no. 12, pp. 1427-1441, Dec 2015 (bib)
	G. Klinker Augmented Reality and Industry (original title: Ubiquitous Augmented Reality) Keynote speech. The 14th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2015), Fukuoka, Japan, Sep. 29, 2015. (bib)
	M. Huber, M. Schlegel, G. Klinker Application of Time-Delay Estimation to Mixed Reality Multisensor Tracking Journal on Virtual Reality and Broadcasting (JVRB), Volume 11, 2014, online (22 pages), http://www.jvrb.org/past-issues/11.2014/3877 (bib)
	C. Resch, P. Keitler, G. Klinker Sticky Projections - A New Approach to Interactive Shader Lamp Tracking The 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014), Munich, Germany, Sep. 10 - 12, 2014, pp. 151-156. (bib)
	U. Eck, F. Pankratz, C. Sandor, G. Klinker, H. Laga Comprehensive Workspace Calibration for Visuo-Haptic Augmented Reality The 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014), Munich, Germany, Sep. 10 - 12, 2014, pp. 123-128. (bib)
	C. Waechter, D. Pustka, G. Klinker Real-Time Monocular People Tracking by Sequential Monte-Carlo Filtering 6th International ACM Conference on Computer Vision / Computer Graphics Collaboration Techniques and Applications (MIRAGE), Berlin, Germany, i4, June 2013 (bib)
	J.-M. Normand, M. Sanchez-Vives, C. Waechter, E. Giannopoulos, B. Grosswindhager, B. Spanlang, C. Guger, G. Klinker, M. Srinivasan, M. Slater Beaming into the Rat World: Enabling Real-Time Interaction between Rat and Human Each at Their Own Scale PLoS ONE, Public Library of Science, Oct. 2012, Volume 7, Issue 10, e48331 (bib)
	M. Huber, M. Schlegel, G. Klinker Temporal Calibration in Multisensor Tracking Setups 9. Workshop der GI-Fachgruppe Virtuelle und Erweiterte Realität (VR-AR 2012), Düsseldorf, Germany, Sept. 2012, pp. 201-212. (bib)
	M. Schlegel Zeitkalibrierung für Augmented Reality Anwendungen Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, December 2011 (online version) (bib)
	M. Huber Parasitic Tracking for Ubiquitous Augmented Reality Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, November 2011 (online version) (bib)
	D. Pustka, M. Huber, C. Waechter, F. Echtler, P. Keitler, J. Newman, D. Schmalstieg, G. Klinker Automatic Configuration of Pervasive Sensor Networks for Augmented Reality IEEE Pervasive Computing, Volume 10, Number 3, July-Sept. 2011, pp. 68-79. (bib)
	P. Keitler Management of Tracking and Tracking Accuracy in Industrial Augmented Reality Environments Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, April 2011 (online version) (bib)
	C. Waechter, E. Artinger, M. Duschl, G. Klinker Creating Passion for Augmented Reality Applications - a Teaching Concept for a Lab Course 6th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA, Nov. 29th - Dec. 01st, 2010, pp. 429-438. (bib)
	P. Keitler, D. Pustka, M. Huber, F. Echtler, G. Klinker Management of Tracking for Mixed and Augmented Reality Systems The Engineering of Mixed Reality Systems, E. Dubois, P. Gray, L. Nigay (eds.), Human-Computer Interaction Series, Springer Verlag, 2010. (bib)
	C. Waechter, M. Huber, P. Keitler, M. Schlegel, D. Pustka, G. Klinker A Multi-Sensor Platform for Wide-area Tracking 9th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Seoul, Korea, Oct. 13-16, 2010, pp. 275-276. (bib)
	L. Gruber, S. Gauglitz, J. Ventura, S. Zollmann, M. Huber, M. Schlegel, G. Klinker, D. Schmalstieg, T. Höllerer The City of Sights: Design, Construction and Measurement of an Augmented Reality Stage Set The Ninth IEEE and ACM International Symposium on Mixed and Augmented Reality, Seoul, Korea, Oct. 13 - 16, 2010, pp. 157-163. (bib)
	D. Pustka, J. Willneff, O. Wenisch, P. Lükewille, K. Achatz, P. Keitler, G. Klinker Determining the Point of Minimum Error for 3DOF Pose Uncertainty Representation The Ninth IEEE and ACM International Symposium on Mixed and Augmented Reality, Seoul, Korea, Oct. 13 - 16, 2010, pp. 37-45. (bib)
	P. Keitler, B. Becker, G. Klinker Management of Tracking for Industrial AR Setups The Ninth IEEE and ACM International Symposium on Mixed and Augmented Reality, Seoul, Korea, Oct. 13 - 16, 2010, pp. 73-82. (bib)
	M. Huber, B. Becker, G. Klinker Location aware computing using RFID infrastructure Int. J. Autonomous and Adaptive Communications Systems, Vol. 3, No. 1, 2010, pp. 23 - 38 (bib)
	R. Morales-Garcia, P. Keitler, P. Maier, G. Klinker An Underwater Augmented Reality System for Commercial Diving Operations OCEANS 2009 MTS/IEEE, Biloxi, Mississippi, USA, November 2009. (bib)
	P. Keitler, F. Pankratz, B. Schwerdtfeger, D. Pustka, , G. Klinker, C. Rauch, A. Chathoth, J. Collomosse , Y. Song Mobile Augmented Reality based 3D Snapshots Sechster Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR, Braunschweig, 18./19. November 2009 (bib)
	C. Waechter, D. Pustka, G. Klinker Vision based People Tracking for Ubiquitous Augmented Reality Applications 8th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Orlando, FL, USA, Oct. 19-22, 2009, pp. 221-222. (bib)
	P. Keitler, F. Pankratz, B. Schwerdtfeger, D. Pustka, , G. Klinker, C. Rauch, A. Chathoth, J. Collomosse , Y. Song Mobile Augmented Reality based 3D Snapshots The 8th IEEE and ACM International Symposium on Mixed and Augmented Reality, Orlando, Florida, Oct. 19-22, 2009, pp. 199-200. (bib)
	M. Huber, M. Schlegel, G. Klinker Temporal Calibration in Multisensor Tracking Setups 8th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2009), Orlando, USA, October 2009, pp. 177-178 (bib)
	P. Keitler, M. Schlegel, G. Klinker Indirect Tracking to Reduce Occlusion Problems 4th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA, December 2008, pp. 224-235. (bib)
	D. Pustka, G. Klinker Dynamic Gyroscope Fusion in Ubiquitous Tracking Environments The Seventh IEEE and ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, Sep. 15 - 18, 2008, pp. 13-20. (bib)
	B. Becker, M. Huber, G. Klinker Utilizing RFIDs for Location Aware Computing 5th International Conference on Ubiquitous Intelligence and Computing, UIC 2008, Oslo, Norway, June 23-25, 2008, pp. 216-228. (bib)
	D. Pustka, M. Huber, G. Klinker Integrating Gyroscopes into Ubiquitous Tracking Environments IEEE Virtual Reality, Reno, Nevada, Mar. 8-12, 2008, pp. 283-284. (bib)
	M. Huber, D. Pustka, P. Keitler, F. Echtler, G. Klinker A System Architecture for Ubiquitous Tracking Environments The Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 13 - 16, 2007, pp. 211-214. (bib)
	D. Pustka, P. Keitler, M. Huber, F. Echtler, G. Klinker Ubiquitous Tracking for Quickly Solving Multi-Sensor Calibration and Tracking Problems Demonstration at the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 14 - 16, 2007 (bib)
	D. Pustka, M. Huber, F. Echtler, P. Keitler UTQL: The Ubiquitous Tracking Query Language v1.0 Technical Report TUM-I0718, Oct. 11, 2007. (bib)
	M. Bauer Tracking Errors in Augmented Reality Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, September 2007 (online version) (bib)
	J. Newman, A. Bornik, D. Pustka, F. Echtler, M. Huber, D. Schmalstieg, G. Klinker Tracking for Distributed Mixed Reality Environments IEEE VR 2007 Workshop on 'Trends and Issues in Tracking for Virtual Environments', Charlotte, NC, USA, Mar 11, 2007 (bib)
	H. Najafi Fast 3D Object Detection and Pose Estimation for Augmented Reality Systems Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, December 2006 (online version) (bib)
	D. Pustka, M. Huber, M. Bauer, G. Klinker Spatial Relationship Patterns: Elements of Reusable Tracking and Calibration Systems The Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, Santa Barbara, CA, USA, Oct. 22 - 25, 2006, pp. 88-97. (bib)
	M. Bauer, M. Schlegel, D. Pustka, N. Navab, G. Klinker Interactive Visualization of the Predicted Accuracy of Marker-Based Optical Tracking Systems Demonstration at the Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, Santa Barbara, CA, USA, Oct. 22 - 25, 2006. (bib)
	M. Bauer, M. Schlegel, D. Pustka, N. Navab, G. Klinker Predicting and Estimating the Accuracy of n-occular Optical Tracking Systems The Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality, Santa Barbara, CA, USA, Oct. 22 - 25, 2006, pp. 43-51. (bib)
	D. Pustka Construction of Data Flow Networks for Tracking in Augmented Reality Applications Dritter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR, Koblenz, 25.-26. September 2006 (bib)
	M. Wagner Tracking With Multiple Sensors Dissertation (PhD thesis), Fakultät für Informatik, Technische Universität München, Apr. 2005 (online version) (bib)
	M. Wagner, S. Hennauer, G. Klinker Easing the Transition Between Multiple Trackers International Symposium on Mixed and Augmented Reality (ISMAR) 2004, Arlington, VA, USA, Nov 2-4, 2004 (bib)
	J. Newman, M. Wagner, M. Bauer, A. MacWilliams, T. Pintaric, D. Beyer, D. Pustka, F. Strasser, D. Schmalstieg, G. Klinker Ubiquitous Tracking for Augmented Reality International Symposium on Mixed and Augmented Reality (ISMAR) 2004, Arlington, VA, USA, Nov 2-4, 2004, pp. 192-201. (bib)
	M. Wagner Distributed Tracking with Multiple Sensors for Augmented Reality 1. Workshop "Virtuelle und Erweiterte Realität", TU Chemnitz, 27. - 28. September 2004 (bib)
	M. Wagner Distributed Tracking with Multiple Sensors for Augmented Reality 1. Workshop "Virtuelle und Erweiterte Realität", TU Chemnitz, 27. - 28. September 2004 (bib)
	D. Pustka Handling Error in Ubiquitous Tracking Setups Master Thesis - Technische Universitaet Muenchen (bib)
	G. Klinker, M. Wagner 2nd Joint Advanced Summer School 2004, Course 3: Ubiquitous Tracking for Augmented Reality Technischer Bericht der TU München, TUM-I0417, June 2004 (bib)
	M. Wagner, A. MacWilliams, M. Bauer, G. Klinker, J. Newman, T. Pintaric, D. Schmalstieg Fundamentals of Ubiquitous Tracking Second International Conference on Pervasive Computing, Hot Spots section, Vienna, Austria, Apr. 18--23 2004 (bib)
	J. Newman, M. Wagner, T. Pintaric, A. MacWilliams, M. Bauer, G. Klinker, D. Schmalstieg Fundamentals of Ubiquitous Tracking for Augmented Reality Technical Report TUM-I0323, Technische Universität München; Technical Report TR-188-2-2003-34, Vienna University of Technology (bib)
	J. Newman, M. Wagner, T. Pintaric, A. MacWilliams, M. Bauer, G. Klinker, D. Schmalstieg Fundamentals of Ubiquitous Tracking for Augmented Reality Technical Report TUM-I0323, Technische Universität München; Technical Report TR-188-2-2003-34, Vienna University of Technology (bib)
	M. Wagner, G. Klinker An Architecture for Distributed Spatial Configuration of Context Aware Applications 2nd International Conference on Mobile and Ubiquitous Multimedia, Norrköping, Sweden, Dec. 11-12 2003 (bib)
	M. Wagner DWARF - Design, Prototypical Implementation and Testing of a Real-Time Optical Feature Tracker Diplomarbeit (bib)
	G. Klinker, T. Reicher, B. Bruegge Distributed User Tracking Concepts for Augmented Reality Applications International Symposium on Aumgented Reality (ISAR 2000), Munich, Germany, 2000, pp. 37-44. (bib)
	G. Klinker Augmented Reality: A problem in need of many computer vision-based solutions NATO Advanced Research Workshop at the 8. International Conference on the Computer Analysis of Images and Patterns (CAIP-99), org. by A. Leonardis and R. Bajcsy. Ljubljana Slowenia, Aug. 29-31, 1999. (bib)
	G. Klinker, D. Stricker, D. Reiners The Use of Reality Models in Augmented Reality Applications Proc. European Workshop on 3D Structure from Multiple Images of Large-scale Environments (SMILE), in combination with ECCV'98, 1998; pp. 275-289. (bib)
	G. Klinker, K. Ahlers, D. Breen, P.-Y. Chevalier, C. Crampton, D. Greer, A. Kramer, E. Rose, M. Tuceryan, R. Whitaker Confluence of Computer Vision and Interactive Graphics for Augmented Reality PRESENCE - Teleoperators and Virtual Environments, Special Issue on Augmented Reality 6(4), 1997, pp. 433-451. (bib)