Improving Depth Perception for Video-Based, Intraoperative Augmented Reality Systems.
[Subtitle]Thesis by:
Advisor: Nassir Navab
Supervision by: Christoph Bichlmeier, Stuart Holdstock
Due date:
Abstract
Augmented reality (AR) is still a relatively young field of research. It concerns the extension of the view on a real scene with virtual objects. Virtual objects can be placed at 3D coordinates respective to the real world. This fact is highly attractive since almost any data from measuring devices can be represented in the real world. This gives the possibility of applying AR to a big set of applications in automobile, media, civil, military and medical industries. For this reason it is a highly interdisciplinary field combining a variety of subjects starting from math and ending with psychology. Medical AR is the target application of this master thesis. This branch of augmented reality gets closer and closer to real life applications with time, giving a possibility to help in different ways during various types of surgeries. It also brings probable ways of performing those surgeries, which were not doable before. Best and newest mathematical methods and algorithms are used for calculations of relations between real scene, video devices and virtual objects. But there still exists a serious bottleneck - the correct and correctly perceivable visualization. In cases of industrial or mechanical AR it might cause serious problems. Misleading visualization in Medical AR, being used for intra-operative guidance, is life-threatening. Medical AR in most cases has to show internal body structure i.e the virtual representation of anatomy from different data sources. Here the keyword is - “internal”. Different instances of medical imaging such as readings of measuring devices such as magnetic resonance (MR) scanner, computer tomography (CT), data received from a Gamma Probe are sources of this virtual information. But this information is not always intra-operatively available, and in case of availability - data sets are usually small in volume. For example in the case of vertebroplasty, which is a minimally invasive spine surgery, a CT scan of small spine segment covering 3 vertebrae is taken. Newest techniques of volume rendering allow to improve the perception of depth. In addition to the fact that they work mostly for big datasets, latter are still visually superimposed over the real body, although all spatial relations and positions are correct. In the case of small datasets even these techniques are not useful at all, since the simpler and the smaller the dataset the less depth cues can be created with the rendering algorithm. Thus new ways of depth perception improvement should be invented, old ones combined and all this ensemble should be applied to a maximum extent, getting closer to human everyday’s life perception. This task leads deeply to science of human perception in general and to depth perception in particular. The current master thesis discusses all possible depth cues and the extent to which they can be used in Medical AR to improve perception of depth. Reasons of other depth cues not being useful are also mentioned. Master thesis also describes a novel method of combining certain cues and gives examples of how this combination can be applied in cases of availability of different input data from a real scene. This new approach is compared to those, which were used previously. Results from a user study are analyzed regarding the enhancement of an intuitive depth perception.
Requirements
If you- know how to programm C++, OpenGL, Shader (GLSL)
- are interested in working on a interdisciplinary project mostly at the hospital
- have interest in medicin in general and surgical applications
Literature
- M. Lerotic et al. pq-space Based Non-Photorealistic Rendering for Augmented Reality Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2007), Brisbane, Australia, October/November 2007.
- C. Bichlmeier, F. Wimmer, S.M. Heining, N. Navab Contextual Anatomic Mimesis: Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality The Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, Nov. 13 - 16, 2007.
- M. Feuerstein, T. Mussack, S.M. Heining, N. Navab Intra-Operative Laparoscope Augmentation for Port Placement and Resection Planning in Minimally Invasive Liver Resection To appear in IEEE Trans. Med. Imag.
- T. Wendler, A. Hartl, T. Lasser, J. Traub, F. Daghighian, S. Ziegler, N. Navab Towards intra-operative 3D nuclear imaging: reconstruction of 3D radioactive distributions using tracked gamma probes Medical Image Computing and Computer-Assisted Intervention, MICCAI, 2007 (to appear), Brisbane, Australia, October 29 - November 2 2007
| Students.ProjectForm | |
|---|---|
| Title: | Improving Depth Perception for Video-Based, Intraoperative Augmented Reality Systems. |
| Abstract: | Augmented reality (AR) is still a relatively young field of research. It concerns the extension of the view on a real scene with virtual objects. Virtual objects can be placed at 3D coordinates respective to the real world. This fact is highly attractive since almost any data from measuring devices can be represented in the real world. This gives the possibility of applying AR to a big set of applications in automobile, media, civil, military and medical industries. For this reason it is a highly interdisciplinary field combining a variety of subjects starting from math and ending with psychology. Medical AR is the target application of this master thesis. This branch of augmented reality gets closer and closer to real life applications with time, giving a possibility to help in different ways during various types of surgeries. It also brings probable ways of performing those surgeries, which were not doable before. Best and newest mathematical methods and algorithms are used for calculations of relations between real scene, video devices and virtual objects. But there still exists a serious bottleneck - the correct and correctly perceivable visualization. In cases of industrial or mechanical AR it might cause serious problems. Misleading visualization in Medical AR, being used for intra-operative guidance, is life-threatening. Medical AR in most cases has to show internal body structure i.e the virtual representation of anatomy from different data sources. Here the keyword is - “internal”. Different instances of medical imaging such as readings of measuring devices such as magnetic resonance (MR) scanner, computer tomography (CT), data received from a Gamma Probe are sources of this virtual information. But this information is not always intra-operatively available, and in case of availability - data sets are usually small in volume. For example in the case of vertebroplasty, which is a minimally invasive spine surgery, a CT scan of small spine segment covering 3 vertebrae is taken. Newest techniques of volume rendering allow to improve the perception of depth. In addition to the fact that they work mostly for big datasets, latter are still visually superimposed over the real body, although all spatial relations and positions are correct. In the case of small datasets even these techniques are not useful at all, since the simpler and the smaller the dataset the less depth cues can be created with the rendering algorithm. Thus new ways of depth perception improvement should be invented, old ones combined and all this ensemble should be applied to a maximum extent, getting closer to human everyday’s life perception. This task leads deeply to science of human perception in general and to depth perception in particular. The current master thesis discusses all possible depth cues and the extent to which they can be used in Medical AR to improve perception of depth. Reasons of other depth cues not being useful are also mentioned. Master thesis also describes a novel method of combining certain cues and gives examples of how this combination can be applied in cases of availability of different input data from a real scene. This new approach is compared to those, which were used previously. Results from a user study are analyzed regarding the enhancement of an intuitive depth perception. |
| Student: | Maxim Kipot |
| Director: | Nassir Navab |
| Supervisor: | Christoph Bichlmeier, Stuart Holdstock |
| Type: | Master Thesis |
| Area: | Computer-Aided Surgery, Medical Augmented Reality |
| Status: | finished |
| Start: | |
| Finish: | |
| Thesis (optional): | |
| Picture: | |