Maxim Kipot

Maxim Kipot E-Mail: Address: Chirurgische Klinik und Poliklinik Innenstadt: NARVIS lab (main) Phone: +49 (89) 5160-3615 Fax: +49 (89) 5160-3630 NARVIS office Phone: +49 (89) 5160-2655 Fax: +49 89 5160-4786 Skype:

Project

Improving Depth Perception for Video-Based, Intraoperative Augmented Reality Systems.
supervised by Christoph Bichlmeier, Stuart Holdstock and Nassir Navab
Augmented reality (AR) is still a relatively young field of research. It concerns the extension of the view on a real scene with virtual objects. Virtual objects can be placed at 3D coordinates respective to the real world. This fact is highly attractive since almost any data from measuring devices can be represented in the real world. This gives the possibility of applying AR to a big set of applications in automobile, media, civil, military and medical industries. For this reason it is a highly interdisciplinary field combining a variety of subjects starting from math and ending with psychology. Medical AR is the target application of this master thesis. This branch of augmented reality gets closer and closer to real life applications with time, giving a possibility to help in different ways during various types of surgeries. It also brings probable ways of performing those surgeries, which were not doable before. Best and newest mathematical methods and algorithms are used for calculations of relations between real scene, video devices and virtual objects. But there still exists a serious bottleneck - the correct and correctly perceivable visualization. In cases of industrial or mechanical AR it might cause serious problems. Misleading visualization in Medical AR, being used for intra-operative guidance, is life-threatening. Medical AR in most cases has to show internal body structure i.e the virtual representation of anatomy from different data sources. Here the keyword is - “internal”. Different instances of medical imaging such as readings of measuring devices such as magnetic resonance (MR) scanner, computer tomography (CT), data received from a Gamma Probe are sources of this virtual information. But this information is not always intra-operatively available, and in case of availability - data sets are usually small in volume. For example in the case of vertebroplasty, which is a minimally invasive spine surgery, a CT scan of small spine segment covering 3 vertebrae is taken. Newest techniques of volume rendering allow to improve the perception of depth. In addition to the fact that they work mostly for big datasets, latter are still visually superimposed over the real body, although all spatial relations and positions are correct. In the case of small datasets even these techniques are not useful at all, since the simpler and the smaller the dataset the less depth cues can be created with the rendering algorithm. Thus new ways of depth perception improvement should be invented, old ones combined and all this ensemble should be applied to a maximum extent, getting closer to human everyday’s life perception. This task leads deeply to science of human perception in general and to depth perception in particular. The current master thesis discusses all possible depth cues and the extent to which they can be used in Medical AR to improve perception of depth. Reasons of other depth cues not being useful are also mentioned. Master thesis also describes a novel method of combining certain cues and gives examples of how this combination can be applied in cases of availability of different input data from a real scene. This new approach is compared to those, which were used previously. Results from a user study are analyzed regarding the enhancement of an intuitive depth perception.

Publications

2009
	C. Bichlmeier, M. Kipot, S. Holdstock, S.M. Heining, E. Euler, N. Navab A Practical Approach for Intraoperative Contextual In-Situ Visualization International Workshop on Augmented environments for Medical Imaging including Augmented Reality in Computer-aided Surgery (AMI-ARCS 2009), London, UK, September 2009 (bib)

Research Projects

Improving Depth Perception and Perception of Layout for In-Situ Visualization in Medical Augmented Reality In-situ visualization in medical augmented reality (AR) using for instance a video see-through head mounted display (HMD) and an optical tracking system enables the stereoscopic view on visualized CT data registered with the real anatomy of a patient. Data can aligned with the required accuracy and the surgeons do not have to analyze data on an external monitor or images attached to the wall somewhere in the operating room. Thanks to a medical AR system like mentioned before, surgeons get a direct view onto and also ”into” the patient. Mental registration of medical imagery with the operation site is not necessary anymore. In addition surgical instruments can be augmented inside the human body. Bringing medical imagery and surgical instruments in the same field of action provides the most intuitive way to understand the patient’s anatomy within the region of interest and allows for the development of completely new generations of surgical navigation systems. Unfortunately, this method of presenting medical data suffers from a serious lack. Virtual imagery, such as a volume rendered spinal column, can only be displayed superimposed on real objects. If virtual entities of the scene are expected behind real ones, like the virtual spinal column beneath the real skin surface, this problem implicates incorrect perception of the viewed objects respective their distance to the observer. The strong visual depth cue interposition is responsible for misleading depth perception. This project aims at the development and evaluation of methods to improve depth perception for in-situ visualization in medical AR. Its intention is to provide an extended view onto the human body that allows an intuitive localization of visualized bones and tissue.

Laparoscope Augmentation for Minimally Invasive Liver Resection In recent years, an increasing number of liver tumor indications were treated by minimally invasive laparoscopic resection. Besides the restricted view, a major issue in laparoscopic liver resection is the precise localization of the vessels to be divided. To navigate the surgeon to these vessels, pre-operative imaging data can hardly be used due to intra-operative organ deformations caused by appliance of carbon dioxide pneumoperitoneum and respiratory motion. Therefore, we propose to use an optically tracked mobile C-arm providing cone-beam computed tomography imaging capability intra-operatively. After patient positioning, port placement, and carbon dioxide insufflation, the liver vessels are contrasted and a 3D volume is reconstructed during patient exhalation. Without any further need for patient registration, the volume can be directly augmented on the live laparoscope video. This augmentation provides the surgeon with essential aid in the localization of veins, arteries, and bile ducts to be divided or sealed. Current research focuses on the intra-operative use and tracking of mobile C-arms as well as laparoscopic ultrasound, augmented visualization on the laparoscope's view, and methods to synchronize respiratory motion.

Active Research Projects