Towards Robust Markerless Tracking: Combining Feature-based and Intensity-based Approaches

We propose a complete real-time model-based tracking system for piecewise-planar objects which combines template-based and feature-based approaches. The contributions are an extension to the ESM algorithm used for template-based tracking and the formulation of a feature-based tracking approach, which is specifically tailored for use in a real-time setting. In order to cope with highly dynamic scenarios, such as illumination changes, partial occlusions and fast object movement, the system adaptively switches between template-based tracking, feature-based tracking and a global initialization phase. The tracking system achieves real-time performance by applying a coarse-to-fine optimization approach and includes means to detect a loss of track.

If you want to see the tracking system in action have a look at this video.

Related Publications

A. Ladikos, S. Benhimane, N. Navab A Real-Time Tracking System Combining Template-Based and Feature-Based Approaches International Conference on Computer Vision Theory and Applications, Barcelona, Spain, March 2007. (bib)