Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Interaction Concept for a Medical Augmented Reality System

Thesis by:
Advisor: Nassir Navab
Supervision by: Tobias Blum


CAMPAR is a medical augmented reality framework we have developed, that has been used for many applications during the last years. The goal of this project is to design and implement a state-based interaction concept for this framework. A state machine will be used to model different states of a medical procedure and the AR system. The current state of the system can be changed by events like keybord input, user gestures, voice input or tracking data. Based on the state e.g. the AR visualization is adapted.

Another field of research where we are very active is workflow analysis. We automatically obtain data during a medical procedure, like the tool that is currently used, the position of the surgeon or biomedical data from the patient. Using statistical models like Hidden Markov Models (HMM) the workflow is modeled based on this data. Using probabilistic inference we can during a running procedure detect the current state. In this project the output of an HMM shall also be used as input for the state machine so that it is possible to build an AR system that is always aware of the current state of a procedure.

Workflow AR Visualization

Action items

  • Investigate interaction concepts of other medical imaging frameworks
  • Investigate use of existing state machine implementations
  • Implement or integrate a state machine into CAMPAR
  • Implement model checking for the state machine
  • Integrate with our probabilistic inference methods

Possible extensions

  • Look a possible modifications towards using ontologies for describing the workflow
  • Integrate an open source speech recognition engine

Title: Interaction Concept for a Medical Augmented Reality System
Director: Nassir Navab
Supervisor: Tobias Blum
Type: DA/MA/BA
Area: Surgical Workflow, Medical Augmented Reality
Status: draft
Thesis (optional):  

Edit | Attach | Refresh | Diffs | More | Revision r1.3 - 06 Apr 2011 - 07:04 - TobiasBlum