Multimodal Consultation System for Patient Education in Plastic Surgery

Student: Patrick Wucherer
Advisor: Nassir Navab
Supervision by: Christoph Bichlmeier

Abstract

In the medical field of breast reconstruction, different methods for patient self-education , quality management and approaches to predict real deformations of breasts already exist, however for the preoperative patient-surgeon discussion, appropriate visualization tools are missing. The possibilities for a surgeon to explain the limitations, risks and opportunities of different procedures of breast reconstruction are very limited. Nowadays the surgeon describes all critical issues using 2D illustrations, patient-based image material, 3D animations and videos. The present diploma thesis presents the initial steps of a long term project for improved patient education integrating an augmented reality system into the clinical workflow of breast reconstruction in plastic surgery. Based on feedback from our medical project partner Dr. med. Maximilian Eder (Klinikum Rechts der Isar), four other plastic surgeons and a commonly used patient education guideline provided by the company proCompliance, an augmented reality framework has been developed, to support the patient education. The motivation of this project is to enhance the patient’s knowledge of treatment options, to reduce decisional conflict, and stimulate patients to be more active in decision making without increasing their anxiety. The motivations are based on empirical research which demonstrates that many patients want to actively participate in the discussion. Furthermore the diploma thesis fits perfectly to the increasingly advocated ideal model of treatment decision-making in the medical talks: the shared decision-making. Main aims of the present project are the development of a realistic physical model for soft tissue and the application of this model to a 3D mesh gained from a 3D scanner in order to simulate the reconstructed breast. In addition, methods are proposed for a smooth image composition of the 3D mesh with the real patient. The movement of the patient by wearing a infrared marker set to be tracked by an ART system 2, influences the real-time deformation of the breast, which is simulated by a Mass-Spring System realized on the GPU. The whole augmented reality visualization is embedded into a framework, which also gives guidance to the surgeon through the shared decision-making process. The developed system is capable of integrating all aspects of any treatment option like risks during the patient-surgeon discussion and might thus protect the liability of a surgeon. I believe that the proposed information system for patient education supports the patient in decision making and helps the surgeon to easily explain details of the surgery and thus should improve the patient-surgeon relationship. It follows the ideal model of integrating the patient as well as the surgeon in a balanced way in the medical talks. Offering the patient as much responsibility as she wants and the surgeon an opportunity to go beyond a role of transferring information to also participate in, but not dominate, the decisionmaking process. The project is supported by KKF funding. Future proposals are planned to finally build a prototype for augmented reality based patient education.

Literature

• Optimization of 3-dimensional imaging of the breast region with 3-dimensional laser scanners. Kovacs L, Yassouridis A, Zimmermann A, Brockmann G, Wöhnl A, Blaschke M, Eder M, Schwenzer-Zimmerer K, Rosenberg R, Papadopulos NA, Biemer E.
• New aspects of breast volume measurement using 3-dimensional surface imaging. Kovacs L, Eder M, Hollweck R, Zimmermann A, Settles M, Schneider A, Udosic K, Schwenzer-Zimmerer K, Papadopulos NA, Biemer E.
• Comparison between breast volume measurement using 3D surface imaging and classical techniques. Kovacs L, Eder M, Hollweck R, Zimmermann A, Settles M, Schneider A, Endlich M, Mueller A, Schwenzer-Zimmerer K, Papadopulos NA, Biemer E.
• Computer aided surgical reconstruction after complex facial burn injuries -- opportunities and limitations. Kovacs L, Zimmermann A, Wawrzyn H, Schwenzer K, Seitz H, Tille C, Papadopulos NA, Sader R, Zeilhofer HF, Biemer E.
• A multigrid framework for real-time simulation of deformable bodies. Joachim Georgii, Rüdiger Westermann
• A Generic and Scalable Pipeline for GPU Tetrahedral Grid Rendering. Joachim Georgii, Rüdiger Westermann
• Physically-Based Visual Simulation on Graphics Hardware. Mark K. Harris, Greg Coombe, Thorsten Scheuermann, Anselmo Lastra
• Elastically Deformable Models. Demetri Terzopoulos, John Platt, Alan Barr, Kurt Fleischer
• An Introduction to GPU Accelerated Surgical Simulation. Thomas Sangild Sorensen and Jesper Mosegard
• Implementing mesh-based approaches for deformable objects on GPU. Guido Ranzuglia, Paolo Cignoni, Fabio Ganovelli, Roberto Scopigno
• Towards a GPU-based simulation framework for deformable surface meshes. Vidya Kotamraju, Shahram Payandeh, John Dill
• Physically Based Deformable Models in Computer Graphics. Andrew Nealen, Matthias Müller, Richard Keiser, Eddy Boxerman, Mark Carlson
• Linear Algebra Operators for GPU Implementation of Numerical Algorithms. Jens Krüger, Rüdiger Westermann
• Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behavior. Xavier Provot
• Mass-Spring Systems on the GPU. Joachim Georgii, Rüdiger Westermann
• Whitepaper Shader Model 3.0 Using Vertex Textures. Philipp Gerasimov, Randima Fernando, Simon Green