If you are interested in image segmentation send an email to .
For the topics on registration write to .
Please include last and first name, matriculation number, current semester, discipline (diploma, master, bachelor) including major, and a desired topic.
Abstract
In clinical practice medical imaging is a crucial component of a large number of applications. Usually not only one image modality (imaging device) is used but several (like Computed Tomography, Magnetic Resonance Imaging, Angiography, Ultrasound, or the real patient), providing different kinds of information to physicians. Moreover, medical image data acquired during diagnosis is often used to aid in operations and intraoperative images are revised for follow-up.
Medical image segmentation will cover the first half of topics in this seminar. Segmentation extracts features from images of different modalities for instance to visualize anatomical structures of interest for diagnosis or operative planning or to overlay features of different modalities.
The second half of seminar topics is on medical image registration. One of the goals of registration is to merge different image modalities in order to improve visual judgement for treatment or diagnosis. For instance, injected instruments can be visualized in 3D or tumor positions can be detected, both without opening the patient.
For participants, a general interest in Medicine and Computer Graphics/Vision would be a good point to start from.
Please bear in mind that the topics on segmentation and registration will be on similar levels!
Schein requirements
As usual:
attendance of the weekly meeting (one time absence allowed, otherwise we'll request a medical certificate or some other very good reason)
active class participation (discussions)
handout (8 - 16 pages)
presentation in the weekly meeting (approximately 60 minutes + discussion)
Timeline
You are required to meet the following deadlines for discussing your work with your supervisor:
Medical images or volumes acquired using different modalities (like CT, MR, Angiography, Ultrasound, PET, SPECT) are often not only analyzed visually by a surgeon but also by more or less automated algorithms trying to emulate human perception. The partitioning of the original data into distinct regions is called "segmentation" and has a wide range of applications:
Segmentation for Treatment Planning: (Semi-)automatic segmentation of tumors can dramatically reduce a surgeons' workload when a complete labeling of some anatomical strucures is needed. Instead of manually editing each slice of a 3D data set, segmentation methods can be applied to segment the structure after a seed point is placed by the surgeon in the data set. This is especially useful for planning an operational intervention or radiation treatment.
Segmentation for Registration: Feature-based registration relies on correspondence of anatomical features which thus have to be extracted beforehand. The registration of data sets by aligning the segmented vascular structure is one example.
Atlas-based labeling: When registering a patient's data set with some pre-segmented standard data, anatomical structures can be quickly labeled for further visualization. This is often done for brain atlantes.
Segmentation for Visualization: For gaining a 3D understanding of CT or MR data, segmentation of the anatomical structure of interest can lead to much better visualizations. For virtual colonoscopy for example, a 3D CT data set of the patients abdomen is acquired and the colon's surface is reconstructed using segmentation. This segmentation now allows a virtual flight through the patients colon by rendering it's surface only. The standard procedure of looking for polyps e.g. can now be done without an invasive colonoscopy. The term "augmented reality" denotes the overlay of medical image data onto the real (optical) view. Since it doesn't make sense to overlay the original, detailed data, one might want to add only specific anatomy which leads to the same segemtation task.
Segmentation for Tracking: Augmenting some data onto a real view requires a pose estimation for the current view. This is usually done by using markers or features which in either case must be detected using a fully automatic segmentation.
The basics and methods for segmentation will be the content of the first part of this seminar.
Registration
Image registration can provide enhanced information from different image modalities. For that, the modalities must be aligned such that anatomical features of one modality can be automatically detected in the other modality (or modalities). Modalities can differ in viewpoint (from where the images were taken), the sensor (e.g. X-ray, Utrasound, or laser scanner), and the time the images were recorded. Depending on which specific anatomy is imaged, or which device has been used for it respectively, registration methods differ in their applicability. Furthermore, if a real patient is to be aligned with preoperative data, one has to evaluate how to acquire data from the real patient that can be registered with the image data. For instance, laser scanners can compute point clouds in space, which are later registered with the preoperative image data. For all this different applications one has to determine the best registration method. For instance, registering a region of the heart is hardly performed on rigid models since the beating heart deforms patient or modalities involved. Thus, non-rigid approaches are used in order to compensate for heart beat. Registration methods are split up into feature-based and intensity-based approaches, the former ones registering on anatomical features (landmarks, markers), the latter ones only on image intensities (e.g. gray values). The above described segmentation methods are often used to extract structure from medical images and register only them in feature-based approaches. Intensity-based registration, for instance, needs data comparison, so filters or other image processing algorithms are applied to the modalities for a suitable sensor comparability. Optimizing an initial guess of the registration parameters is crucial in almost any registration procedure. Again, the most suitable optimization algorithm (e.g. nonlinear methods like Gauss-Newton, Levenberg-Marquardt, or Best Neighbor iteration) must be chosen for a specific application. All these issues of medical image registration will be discussed in the second part of this seminar.
The literature given below will be provided, but you are of course encouraged to look for more ;-). Google Scholar does an excellent job in finding journal papers and articles from conference proceedings.
Image Filtering and Scale-Space Theory
Topic Definition:
Mathematical operations for image filtering
Convolution
Difference to correlation
Derivation of discrete operations from continuous ones
Smoothing filters
Introduction to scale-space theory
Literature / Material:
R.C. Gonzalez and R.E. Woods: Digital Image Processing. Prentice Hall, 2nd edition, 2002. (Sections 3.5, 3.6, and 3.8-4.3; see also the companion web site for downloads)
B. Jähne: Digital Image Processing. Springer, 5th revised and extended edition, 2002. (Chapters 4, 5, and 11)
P. Perona and J. Malik: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629-639, 1990. doi:10.1109/34.56205
G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Section 3.3.1)
T. Lindeberg: Scale-space theory in computer vision. Kluwer, 3rd print, 1997. (Chapter 1)
T. Lindeberg: Scale-space: A framework for handling image structures at multiple scales. available online
Edge Detection and Feature Extraction
Topic Definition:
Discrete edge filters and their construction using mathematical approximations
Filtering in the frequency domain
Anisotropic diffusion filtering
Advance edge filters / linking methods
Canny edge detector
Harris detector
Hough transform
Literature / Material:
R.C. Gonzalez and R.E. Woods: Digital Image Processing. Prentice Hall, 2nd edition, 2002. _(Sections 3.5, 3.7-4.2, 4.4, 10.1, and 10.2.2; see also the companion web site for downloads)
B. Jähne: Digital Image Processing. Springer, 5th revised and extended edition, 2002. (Chapters 4, 12 and Sections 16.5.2-16.5.3)
W.K. Pratt: Digital Image Processing. Wiley, 3rd edition, 2001. (Chapter 15 and Section 17.4.3)
O. Faugeras: Three-Dimensional Computer Vision. A Geometric Viewpoint. MIT Press, 2001. (Chapter 4)
M. Sonka, V. Hlavac, R. Boyle: Image Processing, Analysis, and Machine Vision. PWS, 2nd edition, 1999. (Sections 4.3.3-4.3.5 and 5.2.6)
J. Canny: A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6) pp. 679-698, 1986
C. Harris and M. Stephens: A combined corner and edge detector. In Proc. 4th Alvey Vision Conf., Manchester, pp. 147-151, 1988
Classification and comparison of segmentation methods (user-interaction, prior knowledge, speed, ...)
Thresholding
Region growing
Watersheds (both algorithmic implementations)
Graph cuts
Overall segmentation workflow including pre- and postproscessing (with filters and/or morphological operations)
Literature / Material:
R.C. Gonzalez and R.E. Woods: Digital Image Processing. Prentice Hall, 2nd edition, 2002 _(Sections 10.3-10.5; see also the companion web site for downloads)
B. Jähne: Digital Image Processing. Springer, 5th revised and extended edition, 2002. (Chapter 16)
M. Sonka, V. Hlavac, R. Boyle: Image Processing, Analysis, and Machine Vision. PWS, 2nd edition, 1999. (Chapter 5)
D.L. Pham, C. Xu, and J.L. Prince: Current Methods in Medical Image Segmentation. Annual Review of Biomedical Engineering 2:315-337, 2000. doi:10.1146/annurev.bioeng.2.1.315
Parametric Deformable Models (Snakes)
Topic Definition:
Introduction to snakes
Snake forces
Outline of numerical solution
Literature / Material:
M. Kass, A. Witkin, and D. Terzopoulos: Snakes: Active contour models. International Journal of Computer Vision, 1(4):312–331, 1988. doi:10.1007/BF00133570
S. Osher and N. Paragios: Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003. (Chapter 2)
G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Sections 4.3.1 and 4.3.2)
Level Set and Fast Marching Methods
Topic Definition:
Introduction to and illustration of level sets
Mathematical derivation of level set equation
Outline of numerical solution
Literature / Material:
J.A. Sethian: Level Set Methods and Fast Marching Methods. Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. Cambridge University Press, 2nd edition, 1999 (Chapters 1 and 2; download of introduction, chapter 1, and bibliography)
S. Osher and N. Paragios: Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003. (Chapter 1)
G. Aubert and P. Kornprobst: Mathematical Problems in Image Processing. Springer Series in Applied Mathematical Sciences, 2002. (Section 4.3.3-4.3.5)
Projective Geometry and Transformations in Space
Literature / Material:
Multiple View Geometry in Computer Vision. by Richard Hartley, Andrew Zisserman; chapters 2 and 3
Three-Dimensional Computer Vision. by Olivier Faugeras; chapter 2
Introduction to Registration Methodology
Literature / Material:
J.V. Hajnal, D.L.G. Hill, and D.J. Hawkes (editors): Medical Image Registration. The Biomedical Engineering Series, CRC Press, 2001, chapters 2 and 3
A.G. Brown: A survey of Image Registration Techniques. ACM Computing Surveys, vol.24, pp.326-276, 1992
J.B.A. Maintz and M.A. Viergever: A Survey of Medical Image Registration. Medical Image Analysis, vol.2, pp.1-36, 1998
B. Zitova and J. Flusser: Image Registration Methods: a Survey. Image and Vision Computing, vol.24, pp.977-1000, 2003
Linear and Non-Linear optimization techniques
Literature / Material:
W. Press, B. Flannery, S. Teukolsky, W. Vetterling: Numerical Recipes in C. Cambridge University Press, 1988
R. Hartley, A. Zisserman: Appendix 5 and Appendix 6: Least-squares Minimization and Iterative Estimation Methods, in Multiple View Geometry in Computer Vision. Cambridge University Press, 2nd edition, 2003
P.E. Gill, W. Murray, M.H. Wright: Numerical Linear Algebra and Optimization. Volume 1, Addison-Wesley, Redwood, CA, 1991
P.E. Gill, W. Murray: Algorithms for the solution of the nonlinear least-squares problem. SIAM Journal of Numerical Analysis, 15(5), pp. 977-992, 1978
Feature-Based Registration
Literature / Material:
Topics:
Registration from Features: Pose Estimation, Iterative Closest Point
Medical Image Registration via Surfaces
Medical Image Registration on other features (e.g. gradient information)
Literature:
Pose Estimation:
3D/3D: S. Umeyama: Least-Squares Estimation of Transformation Parameters Between Two Point Patterns, IEEE PAMI, Vol 14, No 4, 1991
2D/3D: R. Haralick, H. Joo, C. Lee, X. Zhuang, V Vaidya, and M. Kim. Pose estimation from corresponding point data. IEEE Trans on Systems, Man and Cybernetics, 19(6):1426--1445, Nov. 1989
2D/3D lines: N. Navab and O.D. Faugeras, Monocular Pose Determination From Lines: Critical Sets and Maximum Number of Solutions, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 254-260, June 1993.
Z. Zhang: Iterative Point Matching for Registration of Free-Form Curves. INRIA, March 1992
M.A. Audette, F.P. Ferrie, T.M. Peters: An Algorithmic Overview of Suface Registration Techniques for Medical Imaging. Medical Image Analysis, vol 4, no. 3, pp 201 - 217, 2000
J. Zhan, A. Rangarajan: A Unified Feature-Based Registration Method for Multimodality Images, Proceedings of the 2004 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Arlington, VA, USA, 15-18 April, 2004
Volume Rendering of Medical Image Data
Literature / Material:
R. Yagel: Classification and Survey of Algorithms for Volume Viewing. 1996
T. Todd Elvins, A Survey of Algorithms for Volume Visualization. Computer Graphics, 26 (3), 1992
P. Heckbert: A Survery of Texture Mapping. IEEE Computer Graphics and Applications, Nov 1986
R. Westermann, T. Ertl: Efficiently using Graphics Hardware in Volume Rendering Applications. ACM SIGGRAPH 1998, pp. 169-177
C. Rezk-Salama: Volume Rendering Techniques for General Purpose Graphics Hardware. PhD-Thesis, University of Erlanden, 2001
Intensity-Based Registration
Literature / Material:
J.V. Hajnal, D.L.G. Hill, and D.J. Hawkes (editors): Medical Image Registration. (Chapter 3)
F. Maes, D. Vandermeulen and P. Suetens: Medical Image Registration Using Mutual Information. IEEE Proceedings Vol. 91 No. 10, October 2003
G.P. Penney, J. Weese, J.A. Little, P. Desmedt, D.L.G. Hill, D.J. Hawkes: A Comparison of Similarity Measures for use in 2-D-3-D Medical Image Registration. IEEE Transactions on Medical Imaging, 17(4), Aug 1998
J.H. Hipwell, G.P. Penney, R.A. McLaughlin, K. Rhode, P. Summers, T.C. Cox, J.V. Byrne, J.A. Noble, D.J. Hawkes: Intensity-Based 2-D-3-D Registration of Cerebral Angiograms. IEEE Transactions on Medical Imaging, 22(11), Nov 2003
H. Livyatan, Z. Yaniv, L. Joskowicz: Gradient-Based 2-D/3-D Rigid Registration of Fluoroscopic X-ray to CT. IEEE Transactions on Medical Imaging, 22(11), Nov 2003
D. Tomazevic, B. Likar, T. Slivnik, F. Pernus: 3-D/2-D Registration of CT and MR to X-ray Images. IEEE Transactions on Medical Imaging, 22(11), Nov 2003
Non-Rigid Approaches
Literature / Material:
J.V. Hajnal, D.L.G. Hill, and D.J. Hawkes (editors): Chapter 13: Nonrigid Registration: Concepts, Algorithms, and Applications; Medical Image Registration. The Biomedical Engineering Series, CRC Press, 2001
W.R. Crum, L.D. Griffin, D.L.G. Hill, D.J. Hawkes: Zen and the art of medical image registration: correspondence, homology, and quality. Neuroimage. 2003 Nov;20(3):1425-37.
In clinical practice medical imaging is a crucial component of a large number of applications. Usually not only one image modality (imaging device) is used but several (like Computed Tomography, Magnetic Resonance Imaging, Angiography, Ultrasound, or the real patient), providing different kinds of information to physicians. Moreover, medical image data acquired during diagnosis is often used to aid in operations and intraoperative images are revised for follow-up. Medical image segmentation will cover the first half of topics in this seminar. Segmentation extracts features from images of different modalities for instance to visualize anatomical structures of interest for diagnosis or operative planning or to overlay features of different modalities. The second half of seminar topics is on medical image registration. One of the goals of registration is to merge different image modalities in order to improve visual judgement for treatment or diagnosis. For instance, injected instruments can be visualized in 3D or tumor positions can be detected, both without opening the patient. For participants, a general interest in Medicine and Computer Graphics/Vision would be a good point to start from.