HandEyeCalibration

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Hand-Eye Calibration

Hand-Eye-Calibration

Hand-Eye-Calibration Spatial Relationship Pattern

Hand-Eye Calibration is the simulataneous computation of two unknown spatial relationships in a circle of spatial relationships.

The Problem

The hand-eye calibration problem first appeared and got its name from the robotics community, where a camera ("eye") was mounted on the gripper ("hand") of a robot. The cameras was calibrated using a calibration pattern. Then the unknown transformation from the robot coordinate system to the calibration pattern coordinate system as well as the transformation from the camera to the hand coordinate system need to be estimated simultaneously.

Application

Robot-Camera-Calibration (also Tracker-Camera Calibration)

This is the standard "hand-eye calibration" problem: Calculate the camera-sensor transformation X by using several measurements Ai, Bi (that give you A,B).

  • Robot Arm with Camera
  • Outside-In Tracking System with Inside-Out Tracking System
  • Camera with Gyroscope

Hand-Eye-Calibration

Tracker Alignment

First estimate hand-eye calibration using standard algorithms, then calculate Y from the estimated parameters. Note that the problem is symetric in X and Y.

Tracker Alignment

We then can either use the Hand-Eye-Calibration methods to estimate X and then Y independently, or use the estimated x to calculate y by closing the loop.

handposeseyposes.gif handeycalibration.gif

Here we see the collected poses of the eye (left) and the hand (middle) as well as the combined poses after calibration (right)

Wrong Uses

Hand-Eye calibration algorithms should not be used when we have a possibility to directly measure one of the two unknown spatial relationships.

handeyedirect.jpg handeyesemidirect.jpg

Possible Solutions from the Literature

Roughly, the methods used can be split up into two parts: Most approaches decompose the matrix X into its rotational and translational part and optimize for first the rotation and then the translation. More bibliography can be found at Keith Prices Computer Vision Bibliography.

Determining first the Rotation then the Translation

  • Y. Baillot, S. Julier, D. Brow, M. Livingston
    A Tracker Alignment Framework for Augmented Reality
    ISMAR 2003, Tokio, Japan
    • claim to provide a "framework" for (a) tracker-to-world registration, (b) tracker-to-tracker registration and (c) locatable-to-display registration. this is not necessarily hand-eye calibration since they focus on Optical-See-Through Displays and therefore probably more related to SPAAM?
    • however, Appendix A shows a solution to the hand-eye calibration problem IDEA! (which is a sub-problem of theirs).
    • cites hand-eye calibration from: F. Park, B. Martin
      Robot Sensor Calibration: Solving AX=XB on the Euclidian Group.
      IEEE Trans. Rob.Aut. 10(5):717-721, October 1994
    • they seem to use some implicit constraints to calculate the calibration with less than the required two movement pairs.

Determining Rotation and Translation simultaneously

  • K. Daniilidis,
    Hand-eye calibration using dual quaternions
    Int. Journ. Robotics Res, 18: 286-298, 1999
    • any rigid transformation can be represented by a rotation φ around some axis and then a translation d along that same axis. This is calles screw.
    • referes to Chassless Theorem
    • uses plücker representation of lines
    • uses dual quaternions to represent the screw
    • a good description of that algorithm IDEA! can be found in Jochen Schmidt, Florian Vogt, Heinrich Niemann,
      Robust Hand-Eye Calibration of an Endoscopic Surgery Robot Using Dual Quaternions
      Pattern Recognition, 25th DAGM Symposium, LNCS 2781, pages 548-556.
      • also discusses the selection of movement pairs for increased numerical stability and the elimination of outliers IDEA!
      • Schmidt, Jochen and Vogt, Florian and Niemann, Heinrich
        Vector Quantization Based Data Selection for Hand-Eye Calibration
        In: Girod, B. ; Magnor, M. ; Seidel, H.-P. (Eds.) : Vision, Modeling, and Visualization 2004 (Vision, Modeling, and Visualization 2004 Stanford, USA). Berlin, Amsterdam : Aka / IOS Press, 2004, pp. 21-28.
    • a second paper by the original author:
      Daniilidis, K., Bayro-Corrochano, E.,
      The dual quaternion approach to hand-eye calibration,
      Proceedings of the 13th International Conference on Pattern Recognition, , Vol.1, 25-29 Aug 1996, p.318-322

  • N. Andreff, R. Horaud, and B. Espiau
    . On-line hand-eye calibration.
    In Second International Conference on 3-D Digital Imaging and Modeling (3DIM’99), pages 430–436, Ottawa, October 1999.

  • H. Zhuang and Y. C. Shiu.
    A noise tolerant algorithm for wrist-mounted robotic sensor calibration with or without sensor orientation measurement
    . In Proceedings of the 1992 lEEE/RSJ International Conference on Intelligent Robots and Systems, volume 2, pages 1095–1100, 1992.
    • _H. Zhuang and Z. Qu_
      A new identification jacobian for robotic hand/eye calibration.
      Systems, Man and Cybernetics, IEEE Transactions on, 24(8):1284–1287, 1994.

  • K. H. Strobl and G. Hirzinger.
    Optimal Hand-Eye Calibration
    Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2006), Beijing, China, pp. 4647-4653, October 2006.
    Presents a calibration method for eye-in-hand systems in terms of a parametrization of a stochastic model. A metric on the group of the rigid transformations SE(3) and the corresponding error model are proposed for nonlinear optimization.

Other Works on Hand-Eye Calibration

Implementation

If you knw of any other (free) implementation of one of the hand-eye calibration algorithms, feel free to add the link here.

MATLAB

An implementation for MATLAB based on the Camera Calibration Toolbox is available from Christian Wengert, who did his PhD at ETH Zürich.

Another implementation based on the Tsai-Lenz papers you can find on Zoran Lazarevic's homepage.

Mathematica

Assume that the poses of the "hand" and the "eye" are given in the following coordinate system (as seen in the figure above):

  • hand poses are given as pose of the hand relative to the tracker coordinate system
  • eye poses are given as pose of the calibration pattern relative to the camera coordinate system
First compute the relative motions
   Hc = Flatten[Table[Table[Ait[[j]].Inverse[Ait[[i]]], {j, i+1, Length[Ait]}], {i, Length[Ait]}], 1];
   Hg = Flatten[Table[Table[Inverse[Bit[[j]]].Bit[[i]], {j, i+1, Length[Bit]}], {i, Length[Bit]}], 1];
and from that the matrix X as
   X = HandEyeTsaiLenz[Hg, Hc]
with (Tsai-Lenz 1988)
   Skew[{x_, y_, z_}] := {{0, -z, y}, {z, 0, -x}, {-y, x, 0}}
   HandEyeTsaiLenz[Hg_, Hc_] := Module[{Pg, Pc, lefthand, righthand},
      Pg = Drop[MatrixToQuaternion[#1], 1] & /@ Hg; 
      Pc = Drop[MatrixToQuaternion[#1], 1] & /@ Hc;
    
      (* Solve for the rotational part *)
      lefthand = Flatten[Skew /@ (Pc + Pg), 1];
      righthand =  Flatten[Pc - Pg, 1];
      Pcg = PseudoInverse[lefthand].righthand;
      Pcg = 2/Sqrt[1 + Pcg.Pcg] Pcg;
      Rcg = (1 - 0.5 (Pcg.Pcg))IdentityMatrix[3] + 0.5 *( (Transpose[{Pcg}].{Pcg}) + Sqrt[4 - Pcg.Pcg] Skew[Pcg]);
    
      (* Solve for the translational part *)
      lefthand = Flatten[ (#[[Range[3], Range[3]]] - 
      IdentityMatrix[3]) & /@ Hg, 1];
      righthand = Flatten[(Rcg.(#[[Range[3], 4]]) & /@ Hc), 1] - Flatten[(#[[Range[3], 4]]) & /@ Hg, 1];
      (* Print[lefthand, ". x =", Transpose[{righthand}]]; *)
      Tcg = PseudoInverse[lefthand].righthand;
      Append[Transpose[Append[Transpose[Rcg], Tcg]], {0, 0, 0, 1}]
    ]

Tips & Tricks

  • Coordinate Systems: Make sure that all your coordinate systems you are using are right-handed (or left-handed, but never mixed). Also have a close look at the direction of the transformation the tracking systems are providing; you might need to invert one of the motions before applying the calibration.


Edit | Attach | Refresh | Diffs | More | Revision r1.37 - 29 Nov 2009 - 14:40 - MarcoFeuerstein

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif