TeachingWs05SegmentationRegistrationHauptseminar

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Hauptseminar: Basics of Medical Image Segmentation and Registration

by Martin Groher, Andreas Keil, and Prof. Dr. Nassir Navab

2 SWS
4 ECTS
Time & Location: Fridays, 2:00-3:30pm in room MI 03.13.010
The seminar language is English.

Participation

If you are interested in image segmentation send an email to .
For the topics on registration write to .
Please include last and first name, matriculation number, current semester, discipline (diploma, master, bachelor) including major, and a desired topic.

Abstract

In clinical practice medical imaging is a crucial component of a large number of applications. Usually not only one image modality (imaging device) is used but several (like Computed Tomography, Magnetic Resonance Imaging, Angiography, Ultrasound, or the real patient), providing different kinds of information to physicians. Moreover, medical image data acquired during diagnosis is often used to aid in operations and intraoperative images are revised for follow-up.

Medical image segmentation will cover the first half of topics in this seminar. Segmentation extracts features from images of different modalities for instance to visualize anatomical structures of interest for diagnosis or operative planning or to overlay features of different modalities.

The second half of seminar topics is on medical image registration. One of the goals of registration is to merge different image modalities in order to improve visual judgement for treatment or diagnosis. For instance, injected instruments can be visualized in 3D or tumor positions can be detected, both without opening the patient.

For participants, a general interest in Medicine and Computer Graphics/Vision would be a good point to start from.

Please bear in mind that the topics on segmentation and registration will be on similar levels!

Schein requirements

As usual:
  • attendance of the weekly meeting (one time absence allowed, otherwise we'll request a medical certificate or some other very good reason)
  • active class participation (discussions)
  • handout (8 - 16 pages)
  • presentation in the weekly meeting (approximately 60 minutes + discussion)

Timeline

You are required to meet the following deadlines for discussing your work with your supervisor:
  • 4-6 weeks before presentation: Outline
  • 2 weeks before presentation: Handout
  • 1 week before presentation: Presentation Slides

Templates

Detailed Overview

Segmentation

Surface-rendered skull from CT data

Medical images or volumes acquired using different modalities (like CT, MR, Angiography, Ultrasound, PET, SPECT) are often not only analyzed visually by a surgeon but also by more or less automated algorithms trying to emulate human perception. The partitioning of the original data into distinct regions is called "segmentation" and has a wide range of applications:

  • Segmentation for Treatment Planning: (Semi-)automatic segmentation of tumors can dramatically reduce a surgeons' workload when a complete labeling of some anatomical strucures is needed. Instead of manually editing each slice of a 3D data set, segmentation methods can be applied to segment the structure after a seed point is placed by the surgeon in the data set. This is especially useful for planning an operational intervention or radiation treatment.
  • Segmentation for Registration: Feature-based registration relies on correspondence of anatomical features which thus have to be extracted beforehand. The registration of data sets by aligning the segmented vascular structure is one example.
  • Atlas-based labeling: When registering a patient's data set with some pre-segmented standard data, anatomical structures can be quickly labeled for further visualization. This is often done for brain atlantes.
  • Segmentation for Visualization: For gaining a 3D understanding of CT or MR data, segmentation of the anatomical structure of interest can lead to much better visualizations. For virtual colonoscopy for example, a 3D CT data set of the patients abdomen is acquired and the colon's surface is reconstructed using segmentation. This segmentation now allows a virtual flight through the patients colon by rendering it's surface only. The standard procedure of looking for polyps e.g. can now be done without an invasive colonoscopy. The term "augmented reality" denotes the overlay of medical image data onto the real (optical) view. Since it doesn't make sense to overlay the original, detailed data, one might want to add only specific anatomy which leads to the same segemtation task.
  • Segmentation for Tracking: Augmenting some data onto a real view requires a pose estimation for the current view. This is usually done by using markers or features which in either case must be detected using a fully automatic segmentation.

The basics and methods for segmentation will be the content of the first part of this seminar.

Registration

Image registration can provide enhanced information from different image modalities. For that, the modalities must be aligned such that anatomical features of one modality can be automatically detected in the other modality (or modalities). Modalities can differ in viewpoint (from where the images were taken), the sensor (e.g. X-ray, Utrasound, or laser scanner), and the time the images were recorded. Depending on which specific anatomy is imaged, or which device has been used for it respectively, registration methods differ in their applicability. Furthermore, if a real patient is to be aligned with preoperative data, one has to evaluate how to acquire data from the real patient that can be registered with the image data. For instance, laser scanners can compute point clouds in space, which are later registered with the preoperative image data. For all this different applications one has to determine the best registration method. For instance, registering a region of the heart is hardly performed on rigid models since the beating heart deforms patient or modalities involved. Thus, non-rigid approaches are used in order to compensate for heart beat. Registration methods are split up into feature-based and intensity-based approaches, the former ones registering on anatomical features (landmarks, markers), the latter ones only on image intensities (e.g. gray values). The above described segmentation methods are often used to extract structure from medical images and register only them in feature-based approaches. Intensity-based registration, for instance, needs data comparison, so filters or other image processing algorithms are applied to the modalities for a suitable sensor comparability. Optimizing an initial guess of the registration parameters is crucial in almost any registration procedure. Again, the most suitable optimization algorithm (e.g. nonlinear methods like Gauss-Newton, Levenberg-Marquardt, or Best Neighbor iteration) must be chosen for a specific application. All these issues of medical image registration will be discussed in the second part of this seminar.

Topics & Schedule (preliminary)

Date Topic Student Material Supervisor
Segmentation
Nov. 25, 2005 Image Filtering and Scale-Space Theory Ivan Stefanov Pronchev Slides, Handout, Images and Matlab Code Andreas Keil
Dec. 2, 2005 Edge Detection and Feature Extraction Zahari Ivanov Doychev Slides, Handout, Images and Matlab Code Andreas Keil
Dec. 9, 2005 ALERT! Cancelled      
Dec. 16, 2005 ALERT! Cancelled      
Jan. 13, 2006 Basic Pixel-, Region- and Edge-Based Approaches
Parametric Deformable Models (Snakes) & Level Set and Fast Marching Methods
Andreas Keil Slides Andreas Keil
Registration
Jan. 20, 2006 Projective Geometry and Transformations in Space & Introduction to Registration Methodology
Linear and Non-Linear Optimization Techniques
Jennifer Schiller
Martin Groher
Slides
Handout
Martin Groher
Jan. 27, 2006 Feature-Based Registration Thomas Jansson Slides
Handout
Martin Groher
Feb. 3, 2006 Volume Rendering of Medical Image Data & Intensity-Based Registration Loren Schwarz Slides
Handout
Martin Groher
Feb. 10, 2006 Non-Rigid Registration Moritz Blume Slides
Handout
Martin Groher

Topic Definitions and Material

The literature given below will be provided, but you are of course encouraged to look for more ;-). Google Scholar does an excellent job in finding journal papers and articles from conference proceedings.

Image Filtering and Scale-Space Theory

Topic Definition:
  • Mathematical operations for image filtering
    • Convolution
    • Difference to correlation
    • Derivation of discrete operations from continuous ones
  • Smoothing filters
  • Introduction to scale-space theory

Literature / Material:

  • R.C. Gonzalez and R.E. Woods: Digital Image Processing. Prentice Hall, 2nd edition, 2002. (Sections 3.5, 3.6, and 3.8-4.3; see also the companion web site for downloads)
  • B. Jähne: Digital Image Processing. Springer, 5th revised and extended edition, 2002. (Chapters 4, 5, and 11)
  • P. Perona and J. Malik: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629-639, 1990. doi:10.1109/34.56205
  • G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Section 3.3.1)
  • T. Lindeberg: Scale-space theory in computer vision. Kluwer, 3rd print, 1997. (Chapter 1)
  • T. Lindeberg: Scale-space: A framework for handling image structures at multiple scales. available online

Edge Detection and Feature Extraction

Topic Definition:
  • Discrete edge filters and their construction using mathematical approximations
  • Filtering in the frequency domain
  • Anisotropic diffusion filtering
  • Advance edge filters / linking methods
    • Canny edge detector
    • Harris detector
    • Hough transform

Literature / Material:

  • R.C. Gonzalez and R.E. Woods: Digital Image Processing. Prentice Hall, 2nd edition, 2002. _(Sections 3.5, 3.7-4.2, 4.4, 10.1, and 10.2.2; see also the companion web site for downloads)
  • B. Jähne: Digital Image Processing. Springer, 5th revised and extended edition, 2002. (Chapters 4, 12 and Sections 16.5.2-16.5.3)
  • W.K. Pratt: Digital Image Processing. Wiley, 3rd edition, 2001. (Chapter 15 and Section 17.4.3)
  • O. Faugeras: Three-Dimensional Computer Vision. A Geometric Viewpoint. MIT Press, 2001. (Chapter 4)
  • M. Sonka, V. Hlavac, R. Boyle: Image Processing, Analysis, and Machine Vision. PWS, 2nd edition, 1999. (Sections 4.3.3-4.3.5 and 5.2.6)
  • J. Canny: A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6) pp. 679-698, 1986
  • C. Harris and M. Stephens: A combined corner and edge detector. In Proc. 4th Alvey Vision Conf., Manchester, pp. 147-151, 1988
  • Hough Transform Applet

Basic Pixel-, Region- and Edge-Based Approaches

Topic Definition:
  • Classification and comparison of segmentation methods (user-interaction, prior knowledge, speed, ...)
  • Thresholding
  • Region growing
  • Watersheds (both algorithmic implementations)
  • Graph cuts
  • Overall segmentation workflow including pre- and postproscessing (with filters and/or morphological operations)

Literature / Material:

  • R.C. Gonzalez and R.E. Woods: Digital Image Processing. Prentice Hall, 2nd edition, 2002 _(Sections 10.3-10.5; see also the companion web site for downloads)
  • B. Jähne: Digital Image Processing. Springer, 5th revised and extended edition, 2002. (Chapter 16)
  • W.K. Pratt: Digital Image Processing. Wiley, 3rd edition, 2001. (Chapter 17)
  • M. Sonka, V. Hlavac, R. Boyle: Image Processing, Analysis, and Machine Vision. PWS, 2nd edition, 1999. (Chapter 5)
  • D.L. Pham, C. Xu, and J.L. Prince: Current Methods in Medical Image Segmentation. Annual Review of Biomedical Engineering 2:315-337, 2000. doi:10.1146/annurev.bioeng.2.1.315

Parametric Deformable Models (Snakes)

Topic Definition:
  • Introduction to snakes
  • Snake forces
  • Outline of numerical solution

Literature / Material:

  • M. Kass, A. Witkin, and D. Terzopoulos: Snakes: Active contour models. International Journal of Computer Vision, 1(4):312–331, 1988. doi:10.1007/BF00133570
  • S. Osher and N. Paragios: Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003. (Chapter 2)
  • G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Sections 4.3.1 and 4.3.2)

Level Set and Fast Marching Methods

Topic Definition:
  • Introduction to and illustration of level sets
  • Mathematical derivation of level set equation
  • Outline of numerical solution

Literature / Material:

  • J.A. Sethian: Level Set Methods and Fast Marching Methods. Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. Cambridge University Press, 2nd edition, 1999 (Chapters 1 and 2; download of introduction, chapter 1, and bibliography)
  • J.A. Sethian's excellent web resource
  • S. Osher and N. Paragios: Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003. (Chapter 1)
  • G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Section 4.3.3-4.3.5)

Projective Geometry and Transformations in Space

Literature / Material:
  • Multiple View Geometry in Computer Vision. by Richard Hartley, Andrew Zisserman; chapters 2 and 3
  • Three-Dimensional Computer Vision. by Olivier Faugeras; chapter 2

Introduction to Registration Methodology

Literature / Material:
  • J.V. Hajnal, D.L.G. Hill, and D.J. Hawkes (editors): Medical Image Registration. The Biomedical Engineering Series, CRC Press, 2001, chapters 2 and 3
  • A.G. Brown: A survey of Image Registration Techniques. ACM Computing Surveys, vol.24, pp.326-276, 1992
  • J.B.A. Maintz and M.A. Viergever: A Survey of Medical Image Registration. Medical Image Analysis, vol.2, pp.1-36, 1998
  • B. Zitova and J. Flusser: Image Registration Methods: a Survey. Image and Vision Computing, vol.24, pp.977-1000, 2003

Linear and Non-Linear optimization techniques

Literature / Material:
  • W. Press, B. Flannery, S. Teukolsky, W. Vetterling: Numerical Recipes in C. Cambridge University Press, 1988
  • R. Hartley, A. Zisserman: Appendix 5 and Appendix 6: Least-squares Minimization and Iterative Estimation Methods, in Multiple View Geometry in Computer Vision. Cambridge University Press, 2nd edition, 2003
  • P.E. Gill, W. Murray, M.H. Wright: Numerical Linear Algebra and Optimization. Volume 1, Addison-Wesley, Redwood, CA, 1991
  • P.E. Gill, W. Murray: Algorithms for the solution of the nonlinear least-squares problem. SIAM Journal of Numerical Analysis, 15(5), pp. 977-992, 1978

Feature-Based Registration

Literature / Material:
  • Topics:
    • Registration from Features: Pose Estimation, Iterative Closest Point
    • Medical Image Registration via Surfaces
    • Medical Image Registration on other features (e.g. gradient information)

  • Literature:
    • Pose Estimation:
      • 3D/3D: S. Umeyama: Least-Squares Estimation of Transformation Parameters Between Two Point Patterns, IEEE PAMI, Vol 14, No 4, 1991
      • 2D/3D: R. Haralick, H. Joo, C. Lee, X. Zhuang, V Vaidya, and M. Kim. Pose estimation from corresponding point data. IEEE Trans on Systems, Man and Cybernetics, 19(6):1426--1445, Nov. 1989
      • 2D/3D lines: N. Navab and O.D. Faugeras, Monocular Pose Determination From Lines: Critical Sets and Maximum Number of Solutions, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 254-260, June 1993.
    • Z. Zhang: Iterative Point Matching for Registration of Free-Form Curves. INRIA, March 1992
    • M.A. Audette, F.P. Ferrie, T.M. Peters: An Algorithmic Overview of Suface Registration Techniques for Medical Imaging. Medical Image Analysis, vol 4, no. 3, pp 201 - 217, 2000
    • J. Zhan, A. Rangarajan: A Unified Feature-Based Registration Method for Multimodality Images, Proceedings of the 2004 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Arlington, VA, USA, 15-18 April, 2004

Volume Rendering of Medical Image Data

Literature / Material:
  • R. Yagel: Classification and Survey of Algorithms for Volume Viewing. 1996
  • T. Todd Elvins, A Survey of Algorithms for Volume Visualization. Computer Graphics, 26 (3), 1992
  • P. Heckbert: A Survery of Texture Mapping. IEEE Computer Graphics and Applications, Nov 1986
  • R. Westermann, T. Ertl: Efficiently using Graphics Hardware in Volume Rendering Applications. ACM SIGGRAPH 1998, pp. 169-177
  • C. Rezk-Salama: Volume Rendering Techniques for General Purpose Graphics Hardware. PhD-Thesis, University of Erlanden, 2001

Intensity-Based Registration

Literature / Material:
  • J.V. Hajnal, D.L.G. Hill, and D.J. Hawkes (editors): Medical Image Registration. (Chapter 3)
  • F. Maes, D. Vandermeulen and P. Suetens: Medical Image Registration Using Mutual Information. IEEE Proceedings Vol. 91 No. 10, October 2003
  • G.P. Penney, J. Weese, J.A. Little, P. Desmedt, D.L.G. Hill, D.J. Hawkes: A Comparison of Similarity Measures for use in 2-D-3-D Medical Image Registration. IEEE Transactions on Medical Imaging, 17(4), Aug 1998
  • J.H. Hipwell, G.P. Penney, R.A. McLaughlin, K. Rhode, P. Summers, T.C. Cox, J.V. Byrne, J.A. Noble, D.J. Hawkes: Intensity-Based 2-D-3-D Registration of Cerebral Angiograms. IEEE Transactions on Medical Imaging, 22(11), Nov 2003
  • H. Livyatan, Z. Yaniv, L. Joskowicz: Gradient-Based 2-D/3-D Rigid Registration of Fluoroscopic X-ray to CT. IEEE Transactions on Medical Imaging, 22(11), Nov 2003
  • D. Tomazevic, B. Likar, T. Slivnik, F. Pernus: 3-D/2-D Registration of CT and MR to X-ray Images. IEEE Transactions on Medical Imaging, 22(11), Nov 2003

Non-Rigid Approaches

Literature / Material:
  • J.V. Hajnal, D.L.G. Hill, and D.J. Hawkes (editors): Chapter 13: Nonrigid Registration: Concepts, Algorithms, and Applications; Medical Image Registration. The Biomedical Engineering Series, CRC Press, 2001
  • W.R. Crum, L.D. Griffin, D.L.G. Hill, D.J. Hawkes: Zen and the art of medical image registration: correspondence, homology, and quality. Neuroimage. 2003 Nov;20(3):1425-37.

TeachingForm
Title: Basics of Medical Image Segmentation and Registration
Professor: Nassir Navab
Tutors: Martin Groher, Andreas Keil
Type: Hauptseminar
Information: Hauptseminar, 2 SWS, 4 ECTS
Term: 2005WiSe
Abstract: In clinical practice medical imaging is a crucial component of a large number of applications. Usually not only one image modality (imaging device) is used but several (like Computed Tomography, Magnetic Resonance Imaging, Angiography, Ultrasound, or the real patient), providing different kinds of information to physicians. Moreover, medical image data acquired during diagnosis is often used to aid in operations and intraoperative images are revised for follow-up. Medical image segmentation will cover the first half of topics in this seminar. Segmentation extracts features from images of different modalities for instance to visualize anatomical structures of interest for diagnosis or operative planning or to overlay features of different modalities. The second half of seminar topics is on medical image registration. One of the goals of registration is to merge different image modalities in order to improve visual judgement for treatment or diagnosis. For instance, injected instruments can be visualized in 3D or tumor positions can be detected, both without opening the patient. For participants, a general interest in Medicine and Computer Graphics/Vision would be a good point to start from.


Edit | Attach | Refresh | Diffs | More | Revision r1.107 - 10 Feb 2006 - 15:19 - MartinGroher

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif