ProjectCAOP

Computer-Aided Ophthalmic Procedures Scientific Director: Prof. Dr. Nassir Navab Contact Person(s): Guillaume Zahnd In industrial collaboration with: Carl Zeiss MEDITEC

Keywords: Registration / Visualization, Segmentation, Medical Imaging, Computer-Aided Surgery, Computer-aided Interventions, Computer Vision, Medical Augmented Reality, Machine Learning for Medical Applications

Abstract

In the current workflow of opthamologic surgeries, the surgeon observes the scene in an indirect way through a microscope while performing the surgery with the necessary high handling precision. During membrane peeling, for example, the surgeon has to grasp an anatomical layer of only 10 μm off the retina without damaging it. However, under this limited microscopic view, it becomes very challenging to infer the distance of the surgical instrument to the retina. Issues such as lens distortion, high level of blurriness and lack of haptic feedback complicate the task further. Recently, an intraoperative version of the Optical Coherence Tomography (OCT) was introduced, which provides the 3D information along a scanning line. On the one hand, this modality provides depth information during the surgery. On the other hand, the device has to be manually positioned to the region of interest which further complicates the current workflow of the surgeon (who already has to manipulate the surgical tool, the handheld light source and the microscope). The main goal of the project is support the surgeon in the current workflow and provide additional information during the surgery via advanced computer vision, visualization and augmented reality algorithms.

Pictures

Figure 1: Typical testing setup

Publications

2020
	J. Weiss, M. Sommersperger, M. A. Nasseri, A. Eslami, U. Eck, N. Navab Processing-Aware Real-Time Rendering for Optimized Tissue Visualization in Intraoperative 4D OCT Medical Image Computing and Computer Assisted Interventions (MICCAI) Lima, Peru, October 2020 (bib)
	A. Tran , J. Weiss, S. Albarqouni, S. Faghihroohi, N. Navab Retinal Layer Segmentation Reformulated as OCT Language Processing Medical Image Computing and Computer Assisted Interventions (MICCAI) Lima, Peru, October 2020 (bib)
2019
	J. Weiss, U. Eck, M. A. Nasseri, M. Maier, A. Eslami, N. Navab Layer-Aware iOCT Volume Rendering for Retinal Surgery Eurographics Workshop on Visual Computing for Biology and Medicine, Brno, Czech Republic, September 2019 (bib)
	M. Zhou, X. Wang, J. Weiss, A. Eslami, K. Huang, M. Maier, C. P. Lohmann, N. Navab, A. Knoll, M. A. Nasseri Needle Localization for Robotic Subretinal Injection based on Deep Learning IEEE International Conference on Robotics and Automation(ICRA), 2019 (bib)
2018
	M. Zhou, M. Hamad, J. Weiss, A. Eslami, K. Huang, M. Maier, C. P. Lohmann, N. Navab, A. Knoll, M. A. Nasseri Towards Robotic Eye Surgery: Marker-free, Online Hand-eye Calibration using Optical Coherence Tomography Images IEEE Robotics and Automation Letters (also presented at IROS 2018) (bib)
	J. Weiss, N. Rieke, M. A. Nasseri, M. Maier, A. Eslami, N. Navab Fast 5DOF Needle Tracking in iOCT International Journal of Computer Assisted Radiology and Surgery / 9th International Conference on Information Processing in Computer-Assisted Interventions (IPCAI), Berlin, Germany, June 2018. (bib)
	J. Weiss, N. Rieke, M. A. Nasseri, M. Maier, C. P. Lohmann, N. Navab, A. Eslami Injection Assistance via Surgical Needle Guidance using Microscope-Integrated OCT (MI-OCT) Proc. of International Society of Imaging in the Eye Conference, Association for Research in Vision and Ophthalmology (ARVO), Honolulu, USA, April 2018 (bib)
2017
	I. Laina, N. Rieke, C. Rupprecht, J. Page Vizcaino, A. Eslami, F. Tombari, N. Navab Concurrent Segmentation and Localization for Tracking of Surgical Instruments Accepted to Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Quebec, Canada, September 2017. A pre-print version is available online at arXiv. The first two authors contribute equally to this paper. (bib)
	J. Page Vizcaino, N. Rieke, D. J. Tan , F. Tombari, A. Eslami, N. Navab Automatic Initialization and Failure Detection for Surgical Tool Tracking in Retinal Microsurgery Workshops Bildverarbeitung fuer die Medizin (BVM), Heidelberg, March 12-14, 2017. (bib)
2016
	H. Roodaki, C. Amat di San Filippo, D. Zapp, A. Eslami, N. Navab A Surgical Guidance System for Big-Bubble Deep Anterior Lamellar Keratoplasty Proceedings of the 19th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Athens, Greece, October 2016 (bib)
	N. Rieke, D. J. Tan , F. Tombari, J. Page Vizcaino, C. Amat di San Filippo, A. Eslami, N. Navab Real-Time Online Adaption for Robust Instrument Tracking and Pose Estimation Proceedings of the 19th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Athens, Greece, October 2016 (bib)
	N. Rieke, D. J. Tan , C. Amat di San Filippo, F. Tombari, M. Alsheakhali, V. Belagiannis, A. Eslami, N. Navab Real-time Localization of Articulated Surgical Instruments in Retinal Microsurgery Medical Image Analysis, vol. 34, 82-100, 2016. The original publication is available online at medicalimageanalysisjournal.com. A free version is available until November 17,2016 (bib)
	N. Rieke, A. Eslami, N. Navab Automatic iOCT Positioning During Membrane Peeling via Real-time High Resolution Surgical Forceps Tracking Proc. of International Society of Imaging in the Eye Conference, Association for Research in Vision and Ophthalmology (ARVO), Seattle, USA, April 2016 (bib)
	M. Alsheakhali, A. Eslami, N. Navab Detection of Articulated Instruments in Retinal Microsurgery Proceedings of International Symposium on Biomedical Imaging (ISBI), April 2016, Prague, Czech Republic (bib)
2015
	N. Rieke, D. J. Tan , M. Alsheakhali, F. Tombari, C. Amat di San Filippo, V. Belagiannis, A. Eslami, N. Navab Surgical Tool Tracking and Pose Estimation in Retinal Microsurgery Proceedings of the 18th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Munich, Germany, October 2015 (bib)
	M. Alsheakhali, A. Eslami, N. Navab Microscopic Surgical Tool Type Detection Proceedings of MICCAI Workshop on Interventional Microscopy (IM), October. 2015, Munich, Germany (bib)
	H. Roodaki, K. Filippatos, A. Eslami, N. Navab Introducing Augmented Reality to Optical Coherence Tomography in Ophthalmic Microsurgery International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
	M. Alsheakhali, M. Yigitsoy, A. Eslami, N. Navab Real Time Medical Instrument Detection and Tracking in Microsurgery Bildverarbeitung für die Medizin, March. 2015, Lübeck, Germany (bib)
	M. Alsheakhali, M. Yigitsoy, A. Eslami, N. Navab Surgical Tool Detection and Tracking in Retinal Microsurgery SPIE, Feb. 2015, Orlando, Florida, USA. (bib)

Team

Contact Person(s)

Dr. Guillaume Zahnd

Working Group

Dr. David Tan

Federico Tombari

Hendrik Burwinkel

Hessam Roodaki

Jakob Weiss

Dr. Nicola Rieke

Location

Technische Universität München Institut für Informatik / I16 Boltzmannstr. 3 85748 Garching bei München Tel.: +49 89 289-17058 Fax: +49 89 289-17059

Visit our lab at Garching.



internal project page

Please contact
Guillaume Zahnd for available student projects within this research project.