Problem statement:Medical Augmented Reality (AR) systems display medical data such as volumetric images and segmented structures co-located with the patient’s anatomy. Such Medial AR applications range from surgery planning  to intra-operative guidance systems . Pre-operative volumetric images such as CT or MRI scans are typically available. In order to correctly augment medical information onto the patient it is essential to accurately determine the transformation between the current pose of the patient, the viewing point of the observer, and the associated medical image data. The required transformations can be estimated by registering the surface (skin) of the patient with a surface extracted from the associated volumetric scans. Depth sensors can be used to reconstruct the environment, for example a patient lying on an table in the operating room. Registration methods perform best if the represented surfaces correspond to each other. It is however challenging to extract only relevant parts from the reconstructed environment, for example the upper body of the patient that is contained in a CT scan. Previous work on segmenting and reconstructing the environment using depth cameras  has shown that automatic segmentation of the reconstructed environment is possible without manual intervention. In particular, by exploiting SLAM-based algorithms, a segmentation of the entire scene can be obtained in a robust and computationally efficient way. Once the relevant surface is segmented from the environment scan, the two surfaces can be registered with each other, which results in the required transformation between a well known world coordinate system and the medical data set. Once such a transform is known, augmentations based on the volumetric information can be rendered correctly.
|Title:||Medical Augmented Reality with SLAM-based perception|
|Director:||Prof. Nassir Navab|
|Supervisor:||Federico Tombari, Ulrich Eck|
|Area:||Computer Vision, Medical Augmented Reality|