ProjectNarvis

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

narvis.jpg

NARVIS - navigated augmented reality visualization system

Advanced visualization is getting increasingly important for the operation room of the future. The increasing number of available medical images must be presented to surgery team in new ways in order to support them rather than overloading with more information. In our project NARVIS we integrate an HMD-based (head mounted display) AR system into the operation room for 3D in situ visualization of computed tomography (CT) images. The final system aims at spinal surgery. The work is in close collaboration with our project partners “Klinikum für Unfallchirgie” at LMU, A.R.T. Weilheim, and Siemens Corporate Research in Princeton. The project is funded by Bayerische Forschungsstiftung.

NARVIS


In a phantom experiment we have used a 3D augmented reality (AR) workspace, in which the user wears a video-feed-through head-mounted display to perform operative tasks in thoracoscopic spine surgery. The set up was used to observe and fuse preoperative 3D CT-data and intraoperative 3D fluoroscopy-data as well as 2D endoscopic images in-situ, i.e. in their actual location, overlaid onto a stereoscopic video view. In-situ visualization of rendered three-dimensional volumes (e.g. implants, vascular and neural structures) facilitates the understanding of the object's three-dimensional structure. Furthermore, the system has the ability to track hand-held surgical instruments in 2 different techniques. Graphically enhanced instruments improve the perception of tool position and orientation in respect to targets in the rendered volume. With an intuitive augmented reality user interface the surgeon controls the visualization of the volumetric data. In the phantom experiment the system has proven to effectively enhance information in minimally-invasive spinal surgery.

HMD-skizze_english_height400.jpg

NARVIS sketch

Group coordinator

Tobias Sielhorst

Related publications

F. Bork, R. Barmaki, U. Eck, K. Yu, C. Sandor, N. Navab
Empirical Study of Non-Reversing Magic Mirrors for Augmented Reality Anatomy Learning
International Symposium on Mixed and Augmented Reality (ISMAR), 2017 (bib)
Alana Da Gama, Thiago Menezes Chaves, Lucas Silva Figueiredo, Adriana Baltar, Ma Meng, N. Navab, Veronica Teichrieb, P. Fallavollita
MirrARbilitation: A clinically-related gesture recognition interactive tool for an AR rehabilitation system
Computer Methods and Programs in Biomedicine (bib)
Xiang Wang, S. Habert, C. Schulte zu Berge, P. Fallavollita, N. Navab
Inverse visualization concept for RGB-D augmented C-arms
Computers in Biology and Medicine (bib)
Ma Meng, P. Fallavollita, S. Habert, Simon Weider, N. Navab
Device and System Independent Personal Touchless User Interface for Operating Rooms
International Conference on Information Processing in Computer-Assisted Interventions (IPCAI), 2016 (bib)
P. Fallavollita, A. Brand, L. Wang, E. Euler, P. Thaller, N. Navab, S. Weidert
An augmented reality C-arm for intraoperative assessment of the mechanical axis - a preclinical study
International Journal of Computer Assisted Radiology and Surgery (bib)
Xiang Wang, S. Habert, Ma Meng, C.-H. Huang, P. Fallavollita, N. Navab
Precise 3D/2D calibration between a RGB-D sensor and a C-arm fluoroscope
International Journal of Computer Assisted Radiology and Surgery (bib)
Ma Meng, P. Fallavollita, Ina Seelbach, A. von der Heide, E. Euler, J. Waschke, N. Navab
Personalized augmented reality for anatomy education
Clinical Anatomy (bib)
P. Fallavollita, L. Wang, S. Weidert, N. Navab
Augmented Reality in Orthopaedic Interventions and Education
Computational Radiology for Orthopaedic Interventions, Springer International Publishing 2016 (bib)
Xiang Wang, S. Habert, Ma Meng, C.-H. Huang, P. Fallavollita, N. Navab
RGB-D/C-arm Calibration and Application in Medical Augmented Reality
International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
Ma Meng, , P. Fallavollita, N. Navab
Natural user interface for ambient objects
International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
N. Leucht, S. Habert, P. Wucherer, S. Weidert, N. Navab, P. Fallavollita
Augmented Reality for Radiation Awareness
International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
S. Habert, Ma Meng, W. Kehl, Xiang Wang, F. Tombari, P. Fallavollita, N. Navab
Augmenting mobile C-arm fluoroscopes via Stereo-RGBD sensors for multimodal visualization
International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
S. Habert, J. Gardiazabal, P. Fallavollita, N. Navab
RGBDX: first design and experimental validation of a mirror-based RGBD Xray imaging system
International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
F. Bork, B. Fuerst, A.K. Schneider, F. Pinto, C. Graumann, N. Navab
Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality
International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
F. Milletari, V. Belagiannis, N. Navab, P. Fallavollita
Fully automatic catheter localization in C-arm images using l1- Sparse Coding
Proceedings of the 17th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Boston, September 2014 (bib)
P. Fallavollita, Alexander Winkler, S. Habert, P. Wucherer, P. Stefan, Riad Mansour, R. Ghotbi, N. Navab
'Desired-View' controlled positioning of angiographic C-arms
Proceedings of the 17th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Boston, September 2014 (bib)
Xiang Wang, S. Demirci, C. Schulte zu Berge, P. Fallavollita, N. Navab
Improved Interventional X-ray Appearance
International Symposium on Mixed and Augmented Reality (ISMAR), 2014 (bib)
Alana Da Gama, P. Fallavollita, Veronica Teichrieb, N. Navab
Motor rehabilitation using Kinect: a systematic review
Games for Health Journal (bib)
Jian Wang, L. Wang, Matthias Kreiser, N. Navab, P. Fallavollita
Augmented Depth Perception Visualization in 2D/3D Image Fusion
Computerized Medical Imaging and Graphics (bib)
X. Chen, Hemal Naik, L. Wang, N. Navab, P. Fallavollita
Video Guided Calibration of an Augmented Reality mobile C-arm
International Journal of Computer Assisted Radiology and Surgery (IJCARS) (bib)
P. Fallavollita
Detection, Tracking and Related Costs of Ablation Catheters in the Treatment of Cardiac Arrhythmias
Cardiac Arrhythmias - Mechanisms, Pathophysiology, and Treatment, Publisher: INTECH, Editor: Wilbert S. Aronow (bib)
P. Stefan, P. Wucherer, Y. Oyamada, Ma Meng, A. Schoch, , , , , M. Weigl, , P. Fallavollita, , N. Navab
An AR Edutainment System Supporting Bone Anatomy Learning
IEEE Virtual Reality (bib)
C. Amat di San Filippo, G. Fichtinger, William Morris, Tim Salcudean, E. Dehghan, P. Fallavollita
Intraoperative Segmentation of Iodine and Palladium Radioactive Sources in C-arm images
International Journal for Computer Assisted Radiology and Surgery (IJCARS) (bib)
P. Thaller, B. Diotte, P. Fallavollita, L. Wang, S. Weidert, E. Euler, W. Mutschler, N. Navab
Video-assisted interlocking of intramedullary nails
Injury, 2013 (bib)
P. Thaller, B. Diotte, P. Fallavollita, L. Wang, S. Weidert, W. Mutschler, N. Navab, E. Euler
Video-navigated targeting unit for interlocking of intramedullary nails – experimental study on a new technique.
Kongress der Gesellschaft für Extremitätenverlängerung und Rekonstruktion, 2013 (bib)
C. Amat di San Filippo, G. Fichtinger, William Morris, Tim Salcudean, E. Dehghan, P. Fallavollita
Declustering n-connected components: an example case for the segmentation of iodine implants in C-arm images
4th International Conference on Information Processing in Computer-Assisted Interventions (IPCAI), Heidelberg, Germany, June 26, 2013 (bib)
S. Weidert, P. Wucherer, P. Stefan, P. Fallavollita, E. Euler, N. Navab
Vertebroplasty medical simulation learning environment
International Society for Computer Assisted Orthopaedic Surgery, Orlando, June 2013 (bib)
S. Pati, O. Erat, L. Wang, S. Weidert, E. Euler, N. Navab, P. Fallavollita
Accurate pose estimation using a single marker camera calibration system
SPIE Medical Imaging, Orlando, Florida, USA, February 2013 (bib)
O. Erat, O. Pauly, S. Weidert, P. Thaller, E. Euler, W. Mutschler, N. Navab, P. Fallavollita
How a Surgeon becomes Superman by visualization of intelligently fused multi-modalities
SPIE Medical Imaging, Orlando, Florida, USA, February 2013 (bib)
Ma Meng, P. Fallavollita, T. Blum, U. Eck, C. Sandor, S. Weidert, J. Waschke, N. Navab
Kinect for Interactive AR Anatomy Learning
The 12th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, Australia, Oct. 1 - 4, 2013. (bib)
Y. Oyamada, P. Fallavollita, N. Navab
Single Camera Calibration using partially visible calibration objects based on Random Dots Marker Tracking Algorithm
The IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Workshop on Tracking Methods and Applications, 2012 (bib)
Jian Wang, P. Fallavollita, L. Wang, Matthias Kreiser, N. Navab
Augmented Reality during Angiography: Integration of a Virtual Mirror for Improved 2D/3D Visualization
The 11th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Atlanta, USA, Nov. 5 - 8, 2012. (bib)
O. Pauly, A. Katouzian, A. Eslami, P. Fallavollita, N. Navab
Supervised Classification for Customized Intraoperative Augmented Reality Visualization
The 11th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Atlanta, USA, Nov. 5 - 8, 2012. (bib)
T. Blum, R. Stauder, E. Euler, N. Navab
Superman-like X-ray Vision: Towards Brain-Computer Interfaces for Medical Augmented Reality
The 11th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Atlanta, USA, Nov. 5 - 8, 2012. The original publication is available online at ieee.org. (bib)
A. Aichert, M. Wieczorek, Jian Wang, L. Wang, Matthias Kreiser, P. Fallavollita, N. Navab
The Colored X-rays
MICCAI 2012 Workshop on Augmented Environments for Computer-Assisted Interventions (bib)
L. Wang, P. Fallavollita, A. Brand, O. Erat, S. Weidert, P. Thaller, E. Euler, N. Navab
Intra-op measurement of the mechanical axis deviation: an evaluation study on 19 human cadaver legs
Proceedings of the 15th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Nice, France, October 2012 (bib)
E. Dehghan, J. Lee, P. Fallavollita, N. Kuo, A. Deguet, Yi Le, C. Burdette, D. Song, J. Prince, G. Fichtinger
Ultrasound-Fluoroscopy Registration for Prostate Brachytherapy Dosimetry
Medical Image Analysis, 2012 (bib)
A. Brand, L. Wang, P. Fallavollita, Philip Sandner, N. Navab, P. Thaller, E. Euler, S. Weidert
Intraoperative measurement of mechanical axis alignment by automatic image stitching: a human cadaver study
International Society for Computer Assisted Orthopaedic Surgery (CAOS), South Korea, June 2012 (bib)
P. Fallavollita
The Future of Cardiac Mapping
Cardiac Arrhythmias - New Considerations, Publisher: INTECH, Editor: Francisco R. Breijo-Marquez (bib)
C. Wachinger, M. Yigitsoy, E. Rijkhorst, N. Navab
Manifold Learning for Image-Based Breathing Gating in Ultrasound and MRI
Medical Image Analysis, Volume 16, Issue 4, May 2012, Pages 806–818. (bib)
J. Traub
New Concepts for Design and Workflow Driven Evaluation of Computer Assisted Surgery Solutions
Dissertation an der Fakultät für Informatik, Technische Universität München, 2008 ( online version available here) (bib)
T. Sielhorst
New Methods for Medical Augmented Reality
Dissertation an der Fakultät für Informatik, Technische Universität München, 2008 ( online version available here) (bib)
T. Sielhorst, Wu Sa, A. Khamene, F. Sauer, N. Navab
Measurement of absolute latency for video see through augmented reality
Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'07), Nara, Japan, November 2007 (bib)
T. Sielhorst, M. Bauer, O. Wenisch, G. Klinker, N. Navab
Online Estimation of the Target Registration Error for n-ocular Optical Tracking Systems
to appear Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2007), Brisbane, Australia, October 2007, pp. 652-659. (bib)
T. Sielhorst, C. Bichlmeier, S.M. Heining, N. Navab
Depth perception a major issue in medical AR: Evaluation study by twenty surgeons
Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI 2006), Copenhagen, Denmark, October 2006, pp. 364-372
The original publication is available online at www.springerlink.com
(bib)
T. Sielhorst, M. Feuerstein, J. Traub, O. Kutter, N. Navab
CAMPAR: A software framework guaranteeing quality for medical augmented reality
International Journal of Computer Assisted Radiology and Surgery, 2006; Vol. 1 Suppl. 1, page 29 - 30.
The original publication is available online at www.springerlink.com
(bib)


Edit | Attach | Refresh | Diffs | More | Revision r1.4 - 27 Jul 2006 - 06:28 - TobiasSielhorst

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif