Multimodal Consultation System for Patient Education in Plastic Surgery
Student: Patrick Wucherer
Advisor: Nassir Navab
Supervision by: Christoph Bichlmeier
In the medical field of breast reconstruction, different methods for patient self-education
, quality management and approaches to predict real deformations
of breasts already exist, however for the preoperative patient-surgeon discussion,
appropriate visualization tools are missing. The possibilities for a surgeon to explain
the limitations, risks and opportunities of different procedures of breast reconstruction are
very limited. Nowadays the surgeon describes all critical issues using 2D illustrations,
patient-based image material, 3D animations and videos.
The present diploma thesis presents the initial steps of a long term project for improved
patient education integrating an augmented reality system into the clinical workflow of
breast reconstruction in plastic surgery.
Based on feedback from our medical project partner Dr. med. Maximilian Eder (Klinikum
Rechts der Isar), four other plastic surgeons and a commonly used patient education guideline
provided by the company proCompliance, an augmented reality framework has
been developed, to support the patient education. The motivation of this project is to
enhance the patient’s knowledge of treatment options, to reduce decisional conflict, and
stimulate patients to be more active in decision making without increasing their anxiety. The motivations are based on empirical research which demonstrates that many
patients want to actively participate in the discussion. Furthermore the diploma thesis fits
perfectly to the increasingly advocated ideal model of treatment decision-making in the
medical talks: the shared decision-making. Main aims of the present project are the
development of a realistic physical model for soft tissue and the application of this model
to a 3D mesh gained from a 3D scanner in order to simulate the reconstructed breast. In
addition, methods are proposed for a smooth image composition of the 3D mesh with the
real patient. The movement of the patient by wearing a infrared marker set to be tracked
by an ART system 2, influences the real-time deformation of the breast, which is simulated
by a Mass-Spring System realized on the GPU. The whole augmented reality visualization
is embedded into a framework, which also gives guidance to the surgeon through
the shared decision-making process. The developed system is capable of integrating all aspects
of any treatment option like risks during the patient-surgeon discussion and might
thus protect the liability of a surgeon.
I believe that the proposed information system for patient education supports the patient
in decision making and helps the surgeon to easily explain details of the surgery and thus
should improve the patient-surgeon relationship. It follows the ideal model of integrating
the patient as well as the surgeon in a balanced way in the medical talks. Offering the patient as much responsibility as she wants and the surgeon an opportunity to go beyond
a role of transferring information to also participate in, but not dominate, the decisionmaking
The project is supported by KKF funding. Future proposals are planned to finally build a
prototype for augmented reality based patient education.
- Optimization of 3-dimensional imaging of the breast region with 3-dimensional laser scanners. Kovacs L, Yassouridis A, Zimmermann A, Brockmann G, Wöhnl A, Blaschke M, Eder M, Schwenzer-Zimmerer K, Rosenberg R, Papadopulos NA, Biemer E.
- New aspects of breast volume measurement using 3-dimensional surface imaging. Kovacs L, Eder M, Hollweck R, Zimmermann A, Settles M, Schneider A, Udosic K, Schwenzer-Zimmerer K, Papadopulos NA, Biemer E.
- Comparison between breast volume measurement using 3D surface imaging and classical techniques. Kovacs L, Eder M, Hollweck R, Zimmermann A, Settles M, Schneider A, Endlich M, Mueller A, Schwenzer-Zimmerer K, Papadopulos NA, Biemer E.
- Computer aided surgical reconstruction after complex facial burn injuries -- opportunities and limitations. Kovacs L, Zimmermann A, Wawrzyn H, Schwenzer K, Seitz H, Tille C, Papadopulos NA, Sader R, Zeilhofer HF, Biemer E.
- A multigrid framework for real-time simulation of deformable bodies. Joachim Georgii, Rüdiger Westermann
- A Generic and Scalable Pipeline for GPU Tetrahedral Grid Rendering. Joachim Georgii, Rüdiger Westermann
- Physically-Based Visual Simulation on Graphics Hardware. Mark K. Harris, Greg Coombe, Thorsten Scheuermann, Anselmo Lastra
- Elastically Deformable Models. Demetri Terzopoulos, John Platt, Alan Barr, Kurt Fleischer
- An Introduction to GPU Accelerated Surgical Simulation. Thomas Sangild Sorensen and Jesper Mosegard
- Implementing mesh-based approaches for deformable objects on GPU. Guido Ranzuglia, Paolo Cignoni, Fabio Ganovelli, Roberto Scopigno
- Towards a GPU-based simulation framework for deformable surface meshes. Vidya Kotamraju, Shahram Payandeh, John Dill
- Physically Based Deformable Models in Computer Graphics. Andrew Nealen, Matthias Müller, Richard Keiser, Eddy Boxerman, Mark Carlson
- Linear Algebra Operators for GPU Implementation of Numerical Algorithms. Jens Krüger, Rüdiger Westermann
- Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behavior. Xavier Provot
- Mass-Spring Systems on the GPU. Joachim Georgii, Rüdiger Westermann
- Whitepaper Shader Model 3.0 Using Vertex Textures. Philipp Gerasimov, Randima Fernando, Simon Green