Rules2011

ISMAR Tracking-Contest
News and Information for Participants

These rules apply to the ISMAR 2011 Tracking-Competition 2011

Environment

  1. The competition takes place in special designated contest ranges. In order to simulate a storage room environment, the room will be separated into different areas by moveable walls, shelves, or similar objects. In particular, there will be no single place from which the whole room can be observed.
  2. The organizers will define a global coordinate frame in the room using geodesic equipment. For registration with the participant’s tracking system, a number of reference points, spread throughout the whole room, will be provided. The coordinates of these reference points are provided as a text file to the participants before the setup phase. The shape of a reference point is given in the following picture:
    round_marker.png
  3. The unit used for all coordinates is meters.
  4. Participants are not allowed to make permanent modifications (e.g. drilling holes) to the room or its equipment.
  5. The room will contain a number of designated “picking areas” from which the contest objects are to be picked. The extent of these areas will be clearly marked. Participants are not allowed to place markers or any other objects into these areas.
  6. In order to provide different levels of difficulty, the objects to pick and the individual picking environments will vary in size and setup.
  7. Participants can assume the availability of reference points in the vicinity of the picking areas in accordance with the individual difficulty. Precise local coordinate frame alignment should be attainable in any case.

Visualization System

  1. The participants are required to implement a suitable visualization to support a picking-task. The organizers will not provide binaries or source code for visualization.
  2. The location of the object to pick must be visualized on a tracked mobile display by highlighting it in a video image or by otherwise overlaying it onto the user’s view.
  3. The location of the object must be visualized by showing a 3D (e.g. sphere) or 2D (e.g. crosshair, circle) object registered at the given coordinates.
  4. If the object to pick is outside the user’s view, a suitable navigation hint (e.g. 2D arrow) should be displayed to indicate the direction to look to find the object.
  5. Any other type of output (e.g. textual) is not allowed. In particular, the visualization must not make use of a map of the environment.
  6. The system must provide a mechanism to select the next object, such as a button or speech input.
  7. The participants are required to provide some means for the organizers to access the live augmented video during the contest run, for example by providing a VGA output. The participants further agree to have this video image publicly displayed. The technical implementation should be coordinated with the organizers.

Hardware Requirements

  1. The display hardware must be mobile such that it can be carried by a single user while performing the picking task. It is not permitted to install display devices (e.g. projectors) in the environment.
  2. There are no other restrictions on the type of tracking or display equipment used. Thus all kinds of tracking methods (for example markerless inside-out, n-ocular outside-in, inertial, etc.) are welcome.

Contest Procedure

  1. Participants will be given some time for the setup and testing.
  2. Details about the schedule will be made available after the registration deadlines.
  3. The setup phase is supervised by the organizers.
  4. For a dry run of their systems, participants will be provided with coordinates of a few selected test objects.
  5. The system will be tested by the organizers after setup, to make sure that it complies with the rules described in section "Visualization System".

Contest Run

  1. At the beginning of the contest run, participants will receive the list of objects on a USB memory stick. The memory stick holds a Unix-format text file named “contest.dat” which contains the 3D coordinates of the objects. For each object, the file contains a line with three floating point numbers, separated by spaces. An example file is provided upon request. Participants are not allowed to display the contents of the file in any other way than described in section "Visualization System".
  2. The contest run is performed by a single person (“the user”) selected by the participant.
  3. Only the mobile equipment carried by the user may be moved during the contest run. All other equipment must stay in place.
  4. During the contest run, the user has to pick the objects in the given order.
  5. If, after an appropriate effort, the participant is unsure about which object to pick, an item can be skipped. Skipped items can be revisited after the last regular item. The order for revisiting has to be kept the original order and any item may be revisited at most once.
  6. All users start the contest run at the same location and have to return to it after having picked all objects.
  7. The contest run is supervised by the organizers who will note the objects that were picked and measure the required time.
  8. The timing starts when the location of the first object is displayed at the starting location and ends when the user has returned to the starting location.
  9. While each contest run is publicly viewable, spectators are not allowed into the actual contest ranges either during or between contest runs.
  10. The organizers will videotape the contest run to present the results to the ISMAR audience at the end of the conference.

Determining the winner

  1. To determine the winner, a point system is used. The team with the highest number of points wins. Points will be allotted according to the following rules.
  2. Correctly picked item score positive points. The number of points depends on the difficulty ratings of the individual picking areas.
  3. Wrongly picked items score negative points. The number of negative points similarly depends on the difficulty rating of the picking area. The overall penalties for wrong items outweigh the positive points for correct items in order to discourage “picking by chance“.
  4. Not picked items score zero points.
  5. Additional bonus points are awarded for the overall top participants in the categories speed and accuracy.
  6. Similarly the requirement of modifying the environment (installation of sensors or fiducals) results in a additional penalty points, based on the extent of the modification.
  7. The exact weighting scheme will be announced before the contest. This includes the influence of the different difficulty ratings as well as the bonus and penalty system.
  8. The concrete difficulty ratings of the individual picking areas are determined and fixed before the contest, but are kept secret until after the last contest run.
  9. In case multiple contestants obtain the same number of points, the overall time required for contest run and for tracking setup are used as for tie breaking.



Edit | Attach | Refresh | Diffs | More | Revision r1.9 - 25 Jul 2011 - 12:56 - ManuelHuber

Lehrstuhl für Computer Aided Medical Procedures & Augmented Reality    rss.gif