Augmented Reality

The vision of medical Augmented Reality is to provide a view into the patient. We have been doing research on Augmented Reality in surgery for many years. Using head-mounted displays we have been working on showing e.g. the tip of an instrument during minimally invasive surgeries. While it is very difficult to develop system that can be used in real surgeries, with our latest development of the magic mirror we have moved towards applications outside surgery.
For more information on our research on medical Augmented Reality have a look at our research page.

Skeleton Tracking

For the magic mirror we have to estimate the pose of a person standing in front of the mirror. We are currently doing this using the Microsoft Kinect. The Kinect is essentially a depth camera that can measure the depth at each image pixel. Based on this depth image, the skeleton of the user can be computed. To obtain this information from the Kinect we are using the OpenNI drivers.

Volume Visualization

For the visualization of bones or other structures we use a 3D image of the user, like CT or MR. These volumes are visualized in 3D using a raycaster which has been developed by Oliver Kutter. As we do not have this data for each person that is standing in front of the mirror, we are using another dataset which is registered to the user. For this we are using the Visible Korean Human dataset.