TeachingWs06BasicMathTools

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Basic Mathematical Tools for Imaging and Visualization

Administrative Info

Lecture by Prof. Nassir Navab
Exercises by Selim BenHimane, Tobias Lasser, Darko Zikic

2+2 SWS, 5 ECTS, Theoretische Informatik, Wahlpflichtfach

The lecture is given in English.

Time & Location

Monday, 14:00 - 15:30
Thursday, 16:00 - 17:30

The most lessons will take place in the room MI 03.013.010.
The programming exercises will take place in the room MI 02.013.008. These sessions will be announced.

Site Content

Announcements

Overview

In order to solve real-world problems in applied engineering areas of computer science, knowledge of basic mathematical tools is essential.

The aim of this lecture is to provide a basic mathematical toolbox for selected topics of Imaging and Visualization.

We will present the basic and most common mathematical techniques and use mostly applications from Computer Vision and Image Processing for illustration. The same mathematical tools, however, are also needed in many other engineering fields like Artificial Intelligence and Machine Learning, Computer Graphics, Robotics, and so on.

The lecture will have three main parts: Basics, Tools and Practise. In the first part, we will give a reminder of linear algebra, analysis, geometry, probability and statistics basics. We go on by presenting the use of these basic concepts in methods such as parameter estimation and optimization. And finally, the students will have the opportunity to gain a deep understanding and hands-on experience of the methods by implementing them and/or using them to solve real-world problems during the exercises.

Course Schedule

Date Topic Detailed Topic Overview / Material
Mon, Oct 16, 2006 Introduction, Motivation and Overview  
Thu, Oct 19, 2006 Linear Algebra Basics
  • 3D space
  • linear spaces
  • linear mappings and matrices

    Literature: [HM1],[GSLA]
  • Mon, Oct 23, 2006 Matlab Introduction (room MI 02.013.008) MATLABWorkshop.pdf
    Thu, Oct 26, 2006 Linear Algebra Basics
  • linear systems and matrices
  • determinants
  • subspaces and basis
  • kernel and rank

    Literature: [HM1],[GSLA]
  • Mon, Oct 30, 2006 Linear Algebra Basics
  • orthogonal bases and matrices
  • change of basis
  • eigenvalues and vectors

    Literature: [HM1],[GSLA]
  • Thu, Nov 2, 2006 no class due to Siemens field trip  
    Mon, Nov 6, 2006 Exercise: Linear Algebra Basics Applications
  • Singular Value Decompostion (SVD) (slides)
  • Pseudo-Inverse

    Literature: [HM1],[GSLA]

    BMexercise01.pdf
  • Thu, Nov 9, 2006 Exercise: Linear Algebra Basics Applications
  • Application of Singular Value Decompostion (SVD)
  • Application of Pseudo-Inverse
  • Image Warping (Backward and Forward Warping)
  • Interpolation Techniques (Nearest Neighbor, Bilinear Interpolation)

    BMexercise02.pdf
    Additional Material (Matlab Stubs, Test Image, Short Description)

    Extra Material, for your interest:
    Short Intro to Homogeneous Coordiantes and Projective Space
  • Mon, Nov 13, 2006 Probability Basics
  • Probability Space (Axioms of Probability, Conditional Probability, Chain Rule, Independence, Bayes’ Rule)
  • Random Variables (Discrete Random Variables, Probability Density Function (PDF), Cumullative Probability Distribution Function (CDF), Continuous Random Variables)
  • Expectation of a Random Variable
  • Variance and Standard Deviation

    Literature: [PRML]
  • Thu, Nov 16, 2006 Exercise (room MI 02.013.008) Matlab Exercise
    Mon, Nov 20, 2006 Probability Basics
  • Extension to n Dimensions (RV, CDF, PDF, marginal distribution)
  • Gaussian Distribution
  • Covariance
  • Principal Component Analysis (PCA)
  • Thu, Nov 23, 2006 Probablitity Basics
  • Covariance and Correlation
  • Principal Component Analysis (PCA) (PCA Tutorial by Lindsay I Smith.)
  • Bayesian Estimation, Maximum a Posteriori, Maximum Likelihood ([AIMA] Chapter 20, available online)
  • Information Theory Tools: Entropy, Mutual Information ([PRML], [ITIL] Chapter 8, avaliable online)

    Lecture Slides
    Additional Material: Simple PCA Example in Matlab
  • Mon, Nov 27, 2006 Analysis Basics (Derivatives of Multivariate Functions)
  • Partial Derivatives
  • Total Derivatives
  • Minima and Maxima

    Literature: [HM1]
  • Thu, Nov 30, 2006 Analysis Basics (Derivatives of Multivariate Functions)
  • Optimizing with constraints
  • Taylor Expansion
  • Application to Image Processing

    Literature: [HM1]
  • Mon, Dec 4, 2006 Intermediate Exam results
    Thu, Dec 7, 2006 no class due to Dies Academicus  
    Mon, Dec 11, 2006 Exercise and Intermediate Correction  
    Thu, Dec 14, 2006 Basic Image Filtering and Similarity Measures  
    Mon, Dec 18, 2006 Function Minimization: Introduction & Theoretical Background
  • Definite Positive
  • Convexity
  • Minimum definition
  • Gradient and Hessian

    Literature: [NMDS][POGMW][NPB][COBV][OTCBG][NRPTVF]
  • Thu, Dec 21, 2006 Function Minimization: Minimum condition
  • Necessary conditions
  • Sufficient conditions

    Literature: [NMDS][POGMW][NPB][COBV][OTCBG][NRPTVF]
  • Mon, Dec 25, 2006 no class due to Christmas holidays  
    Thu, Dec 28, 2006 no class due to Christmas holidays  
    Mon, Jan 1, 2007 no class due to Christmas holidays  
    Thu, Jan 4, 2007 no class due to Christmas holidays  
    Mon, Jan 8, 2007 Function Minimization: Unidimensional minimization
  • Golden number and Fibonacci sequence methods
  • Newton and secant methods
  • Line search method

    Literature: [NMDS][POGMW][NPB][COBV][OTCBG][NRPTVF]
  • Thu, Jan 11, 2007 Non-Linear Minimization
  • Gradient method

    Literature: [NMDS][POGMW][NPB][COBV][OTCBG][NRPTVF]
  • Mon, Jan 15, 2007 Non-Linear Minimization
  • Conjugate-Gradients method

    Literature: [NMDS][POGMW][NPB][COBV][OTCBG][NRPTVF]
  • Thu, Jan 18, 2007 no class due to Weather Storm  
    Mon, Jan 22, 2007 Non-Linear Minimization
  • Newton + Quasi-Newton + Levenberg-Marquardt methods
  • Simplex

    Literature: [NMDS][POGMW][NPB][COBV][OTCBG][NRPTVF]


    Homework: Template-based Tracking
  • description
  • test sequence: carback.zip (38MB)
  • Thu, Jan 25, 2007 Minimization of Functionals
    (Calculus of Variations Basics)
  • Introduction and Overview
  • Modelling Examples
  • slides
  • Mon, Jan 29, 2007 Minimization of Functionals
    (Calculus of Variations Basics)
  • Derivation of Euler-Lagrange Equations for the Model Problem
  • Results for standard modifications of the Model Problem
  • Examples
  • Handout: variational basics
  • Thu, Feb 1, 2007 Minimization of Functionals
    (Calculus of Variations Basics)
  • Case Study: Deformable Registration using the variational approach (Slides)
  • Mon, Feb 5, 2007 Exam (a A4 paper written on both sides with formula allowed) results
    Thu, Feb 8, 2007    

    Literature

    Linear Algebra:

    • [GSLA] Introduction to Linear Algebra, Gilbert Strang. Wellesley Cambridge Pr, 3rd edition, March 1, 2003.

    Linear Algebra and Analysis Basics with many Applications (in German):

    • [HM1] Höhere Mathematik, Kurt Meyberg and Peter Vachenauer. Springer, 6 edition, 2001.
    • [HM2] Höhere Mathematik 2, Kurt Meyberg and Peter Vachenauer. Springer, 4 edition, 2001.

    Probability Therory Basics:

    • [AIMA] Artificial Intelligence: A Modern Approach, Stuart Russell and Peter Norvig. ( Some chapters available online. )
    • [ITIL] Information Theory, Inference, and Learning Algorithms, David MacKay. ( Book available online for on-screen viewing. )
    • [PRML] Pattern Recognition and Machine Learning, Christopher M. Bishop. Springer, 2006. ( Especially Chapters 1 and 2 are of interest for the lecture. )
    • [MAKBM] Multivariate Analysis (Probability and Mathematical Statistics), J. T. Kent, J. M. Bibby, und K. V. Mardia

    Analysis Basics

    • [HM1] Höhere Mathematik, Kurt Meyberg and Peter Vachenauer. Springer, 6 edition, 2001.

    Function Minimization

    • [NMDS] Numerical Methods for Unconstrained Optimization and Nonlinear Equations, J. Dennis & R. Schnabel. Prentice Hall, 1983.
    • [POGMW] Practical Optimization, P. Gill, W. Murray & M. Wright. Academic Press, 1981.
    • [NPB] Nonlinear Programming, D. Bertsekas. Athena Scientific, 1999.
    • [NRPTVF] Numerical Recipes in C, W. Press, S. Teukolsky, W. Vetterling, B. Flannery. Cambridge University Press, 2002. ( Book available online. )
    • [COBV] Convex Optimization, S. Boyd & L. Vandenberghe. Cambridge University Press, 2004. ( Book available online. )
    • [OTCBG] Optimization toolbox for use with MATLAB: User’s guide, T. Coleman, M. A. Branch & A. Grace. The Mathworks Inc., 2006. ( Book available online. )



    internal


    TeachingForm
    Title: Basic Mathematical Tools for Imaging and Visualization
    Professor: Prof. Nassir Navab
    Tutors: Selim BenHimane,Tobias Lasser,Darko Zikic
    Type: Lecture
    Information: 2+2 SWS, 5 ECTS, Theoretische Informatik, Wahlpflichtfach
    Term: 2006WiSe
    Abstract: In order to solve real-world problems in applied engineering areas of computer science, knowledge of basic mathematical tools is essential. The aim of this lecture is to provide a basic mathematical toolbox for selected topics of Imaging and Visualization. We will present the basic and most common mathematical techniques and use mostly applications from Computer Vision and Image Processing for illustration. The same mathematical tools, however, are also needed in many other engineering fields like Artificial Intelligence and Machine Learning, Computer Graphics, Robotics, and so on. The lecture will have three main parts: Basics, Tools and Practise. In the first part, we will give a reminder of linear algebra, analysis, geometry, probability and statistics basics. We go on by presenting the use of these basic concepts in methods such as parameter estimation and optimization. And finally, the students will have the opportunity to gain a deep understanding and hands-on experience of the methods by implementing them and/or using them to solve real-world problems during the exercises.


    Edit | Attach | Refresh | Diffs | More | Revision r1.54 - 26 May 2009 - 11:57 - DarkoZikic

    Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif