DWARF - Distributed Wearable Augmented Reality Framework
Technische Universität München,
Chair for Applied Software Engineering
Prof. Bernd Bruegge Ph.D.,
Prof. Gudrun Klinker, Ph.D.
(view as PDF file)
The Chair for Applied Software Engineering at the
Technische Universität München is doing research on
the field of
Augmented Reality systems.
Augmented Reality (AR) is a technology by which a user?s view
of the real world is augmented with additional information
from a computer model. Users can work with
and examine real world objects while receiving additional
information about those objects or the task at
hand. Rather than pulling the user into the computer?s
virtual world,
Augmented Reality brings information into the user?s
real world, thereby building upon people?s visual and
spatial skills.
Augmented Reality constitutes a very promising new
user interface concept for many applications, e.g., in
medicine, exterior construction, interior design, the assembly,
maintenance, and repair of complex technical
objects, and games.
From a software architecture point of view
Augmented Reality systems
can be divided into a set of components that each
contribute a dedicated functionality to the whole system.
The task was to find a solution where rapid prototyping
with several researchers on different computers
should be possible. This way we wanted to accommodate
the heterogeneous infrastructure of the chair
consisting of several Intel-based PCs and laptops with
Windows and Linux, Apple workstations and laptops,
and Compaq iPAQ PDAs. Additionally it should be
possible to integrate third party components such as
external trackers into the system. The solution to these
requirements a framework for component-based peerto-
peer systems, called Dwarf. We model Augmented
Reality applications as distributed systems which form
spontaneously from mobile and stationary components.
Such a system can reconfigure itself at runtime by exchanging
components and changing component configurations.
As communication infrastructure we chose
CORBA.
Architecture.
The Dwarf framework is based on
the concept of collaborating distributed services. The
services are interdependent and expose their requirements,
called Needs, and others, called Abilities, with
the help of service managers. On each network node of
a Dwarf system, there is one service manager; there
is no central component. The service manager controls
its local services and maintains descriptions of them.
Each service manager cooperates with the others in the
network to set up connections between services. Each
service has an xml description of its Abilities, Needs
and connectors (communication protocols).
This concept extends the approach of the CORBA
Component Model (CCM) of required and offered interfaces
respectively event sinks and sources towards
more flexibility in the inter-service communication and
is part of the research efforts.
Each Ability has a set of attributes describing
quality-of-service parameters of that service. Likewise,
each Need specifies a predicate about the quality of service
it expects. This predicate is used by the service
manager to select abilities that can provide a sucient
quality of service to satisfy a given need. This predicate
can also be used at runtime to ensure that the
desired quality of service is still provided.
Implementation.
The stationary computers are connected
together using standard 100 megabit ethernet;
the mobile computers (laptops and palmtop) are connected
using IEEE 802.11b wireless ethernet.
The
Dwarf services are realized as separate processes
and threads within single processes. Distributed
middleware, consisting of CORBA and several Dwarf
service managers, connect the services together.
Upon startup, each service registers itself, via
CORBA, with its service manager running on the local
machine, which listens for connections on a well-known
TCP port. The middleware is lightweight enough to
run on the Linux-based palmtop as well as the fullsized
machines.
The service managers running on the different machines
communicate with one another using SLP and
CORBA and set up connections between the services.
The services then use CORBA method calls or CORBA
notification service events to communicate.
The format of communication between the services
is specified in CORBA IDL. Some services have
CORBA interfaces with specified methods that are
called by other services; most, however, communicate
using CORBA structured events.
The implementation of the Dwarf service manager
uses the open-source OmniORB CORBA implementation,
which is also used for the services written in
C++. Additionally, it uses OmniNotify, a CORBA Notification
Service implementation based on OmniORB.
Both OmniORB and OmniNotify were cross-compiled
to Linux StrongARM so that the middleware could
run on the palmtop. Services written in Java use
OpenORB, an open-source Java CORBA implementation,
and JavaORB, its predecessor, which was necessary
for the Java 1.1 virtual machine needed to control
the Cortona VRML browser under Windows.
To find services managed by other service managers,
the service managers use OpenSLP, an implementation
of the Service Location Protocol SLP. This allows
services to be found based on type and named
attributes. OpenSLP can evaluate predicates based on
boolean expressions, including wild cards, which is used
to connect the matching needs and abilities based on
their predicates and attributes.
Tool support.
As a general-purpose monitoring and
debugging tool, we implemented a tool that presents
a graphical view of the network of interconnected services
that dynamically changes when the system con-
figuration changes. It also allows developers to view
arbitrary event streams in the system, which proved
invaluable for debugging.
Using Dwarf, we have previously built Pathfinder,
an experimental system for campus navigation; Fatamorgana
for the visualization of prototype automobile
designs; Tramp for automobile repair;
FixIt? for
machine maintenance; and Paarti, prototype vehicle
construction. We also built Sheep, a multiplayer collaborative
game (figure 1) with tangible user interfaces.
Finally, new projects based on
Dwarf are underway.
At the time of writing, these include
Archie, a collaborative
architectural design system,
Heart, a system
for cardiac surgery, and Pong, a set of simple Augmented
Reality games authored with a Python-based
scripting interface.
Benefits
Our project is the first that uses CORBA to realize
an
Augmented Reality system. Since that first prototype
in 2000 several others have been built based on
the described concept. To use CORBA as the underlying
middleware had several advantages over alternatives
such as socket communication, Java RMI, SOAP,
DCOM, or proprietary solutions.
Platform and language independence. CORBA allows
us to develop our services in a broad range of
programming languages. In particular, we use C++,
Java, and Python. With CORBA we are free to chose
the development language from case to case. Usually
we chose programmin language and operating system
because we need to build against a third-party library
to implement a service. Currently we have a mixture of
services deployed on workstations and laptops running
Apple Mac OS X, Linux, and Microsoft Windows. We
even have services running on a Compaq iPAQ with
Linux.
Rapid prototyping.
As we used the concepts of the
Dwarf framework, all developers were forced to use
components running as separate processes. This lead
to a clear and lean definition of the communication
between different components in CORBA IDL files.
In addition, the relationship between components of a
running system could be displayed dynamically. This
allowed us the very late integration of several student
developers in the overall process without the need for
lengthy documentation of the current system state, a
simple look at the debugging tool suced. In fact,
several people unfamiliar with the overall system were
given and completed successfully self-contained tasks
such as adding a collision detection component two
weeks before the final deadline of the overall system.
Use available hardware. CORBA is running on every
hardware and operating system that is already available
at our chair. So we can concentrate on developing
the service functionality instead of thinking about the
best hardware/operating system combination for a new
system.
Integration of third-party hard- and software.
The
implementation of
Augmented Reality systems is very
laborious if every component should be built from
scratch. On the other side, there are several components
available from other research groups that could
be reused. Quite often these components require a
fixed runtime environment, porting would be to costly.
CORBA helps us to develop wrappers around these
components in the required programming language and
operating system.
Remote monitoring.
As a general-purpose monitoring
and debugging tool, we implemented a tool called
the Dwarf Interactive Visualization Environment. In
analogy to visual programming interfaces in systems
such as AVS, this tool presents a graphical view of the
network of interconnected services that dynamically
changes when the system configuration changes. It also
allows developers to view arbitrary event streams in the
system, which proved invaluable for debugging. In addition,
the tool let us explain the running system to
technically interested spectators and illustrate the distributed
nature of Dwarf.
