Documentation of work due to date:

29.10.2004

• Thesis proposal coming soon
• Trying to install DWARF on Fedora Core 2
• Test implementation of NFT ARToolkit and Inertial to look at movement data
• Interface of Magnetic Tracker
• Idea of a NFT / ARToolkit Simulator for user study and simulation, but I still have to decide if I want to implement it in DWARF or under Windows (I guess Mark would appreciate rather windows that linux/DWARF).

07.10.2004 - 14.07.2004: SORRY I AM ON THE FIJIS

06.10.2004

• Evalualtion of results
  • Is the performance of a dynamically configured NFT-Inertial combination really better and more robust (in terms of speed, reinitializations and user response).
  • Idea: Analyse a recorded video sequence: without inertial information and configuration <--> with configured inertial. Hopefully the performance is better <img src="http://campar.in.tum.de/twiki/pub/TWiki/SmiliesPlugin/smile.gif" alt="smile" title="smile" border="0" /&

• User study on "Magic Book"
  • In order to get rotations as well as translations I will use a 6DOF magnetic tracker
  • Expected results: Common movements (rotations) to configure the NFT - Inertial combination. These study should give answers to the question: How do people use the magic book? What is a fast movement? Are there approaches to optimize the performance of the magic book using this information?

• Implementation of Inertial / NFT Combination
  • The update rate of inertial is higher (120 fps) compared to the update rate of NFT (~ 6-10 fps). During the processing of one NFT frame inertial data is gathered. At the beginning of the processing of a NFT frame the inertial data object (an object that gathers and refines the inerial data) is queried. Due to this data the search window and the template size is adjusted.
  • You could say that serveral inertial frames are collected and synchronized to one NFT frame.
  • These collected frames are compared to previous ones and the change of movement is extracted (e.g movement decreases -> smaller search window)
  • Question right now: What is a fast movement in terms of inertial data? I try to figure out a mapping (function) between collected frame data and grade of movement (e.g FRAME_DATA -> HIGHER MOVEMENT COMPARED TO PREVOIUS FRAME_DATA -> WINDOWSIZE+=5).
  • Inertial Drift: Due to the fact that I only need to compare frame to frame (no absolute information) I think I don't have take care about a inertial correction (like kalman-filter, correction function using image data).

04.10.2004

• Decision concerning NCC: Improvements possible (Paper: Fast Normalized Cross Correlation, J.P. Lewis, Industrial Light & Magic). FFT for the convolution of the image and the template and a recursive algorithm for the calculation of the image energy is used. But I won't go deeper into this.

01.10.2004

• A bottleneck of the NFT version is the computational expensive preprocessing. Right now I am looking how to improve the template matching using Normalized Cross Correlation. It is possible to improve the performance using FFT and recursive algorithms. But I still don't know if I will spend more time on this issue.

29.09.2004

• Right now I am documenting the source code of the NFT ARToolkit version. I am using doxygen to generate a documentation that helps to get along with the new version.
• When I am familiar with the main steps of the computer vision algorithms I will evaluate the input parameters given to the process. These parameters influence robustness, computation time and therefore the usability of NFT. Trade-offs have to be made!
• Next step could be to adopt these parameters to the context. One example: If we somehow (e.g inertial) can predict, that no movement is going to happen (due to information of former processing steps) we can assume that there are almost no changes within two video-frames. We can use a small window for template matching now. On the other hand if we can predict much movement in the next frame we need a large search window to detect the feature point again.

25.09.2004

• Algorithmic principles of the ARToolkit and the NFT-ARToolkit (best idea would be to write a short introduction into the NFT version within the next weeks)
• Started to put some documentation into the source code of the NFT version

Idea:

• The ARToolkit is not very robust concering movements (because of restriction of the template matching method; a trade-off between realtime and robustness was made by Kato)
• By putting an additional information source for movements, a gyro for example, one could predict possible movements and adopt the template matching method. Possible questions: Does it affect the usability of a realtime application? Is this possible with a 3DOF tracker?
• Some things I had a look on that might be useful: Sensor Fusion, Kalman Filter, SVC, NCC, Hidden Markov Models, The Open-CV Toolkit (Intel), Intertrax API, different Tracking Technologies.
• Usablity: It might be interesting how people use for example the "Magic Book". The Magic Book is an application using the NFT-ARToolkit developed at the Hitlab NZ. It might be possible to extract certain gestures, somehow recognize and adopt to them during runtime.

01.09.2004

Ideas for Natural Feature Tracking. Kato coded a new version of the ARToolkit based on Natural Feature Tracking (any 2D planar shape instead of markers)

• Augmented Reality and Natural Feature Tracking
• Algorithms for NFT
• Integration in DWARF
• Hybrid approach for a certain application domain (e.g a car detecting traffic signs) using NFT and other technologies
• Usability studies on NFT
• Integration in DWARF
• What about 3D object recognition?

-- FelixLoew - 05 Oct 2004