Viewer Feature Requests

This is the call for a redesign of the viewer component; this document is my idea of how the viewer should be used (and usable) from outside. Feel free to discuss the whole thing with me. -- MartinBauer - 31 Oct 2003

Since this discussion has siginficantly slowed down there is now a new RefactoringTheViewer page we're the implementation of (some) of the features discussed here is outlined. Feel free to contribute.

Consolidate SceneData and UserAction^? Events

The IDL files for SceneData and UserAction^? Events need to be restructured. In general, it shoul not be necessary to "select" an object at all. If i create an Object with some ThingID^?, and then send out PoseData referring to that ThingID^?, this should be by default connected to that object. If i want to attach some object to some other object, then i have to move it around in the scenegraph. This is where the new MoveObject command is used for.

There are some mayor problems with this approach:

• In applications like ProjectArchie one might want to create, walls leave them alone for a while and somewhen later select one to modify some properties or move it around. Therefore it must be possible (and mandatory) to select and deselect Scenegraph objects.
• Not all objects that are in the scenegraph correspond to one ThingID^? in PoseData Events. Imagine one single pointing device that is used to move & modify all objects in the scene (something like a mouse pointer). -- OtmarHilliges - 01 Nov 2003

To make my point more clear here: I fully understand the Use Case that leads to the selection and deselection mechanism. But in general, what we do here is to move an object in the scenegraph from one topological position to another. Therefore i propose to really make that change in the scenegraph. When i have a pointing device that should be used to move some wirtual object around, this means that for some time the pose of that virtual object is relative to the pointing device. This means that the "Select" action is in fact implemented by moving that object in the scenegraph to a new position relative to the pointing device. The incoming PoseData for the pointing device then moves that object as well. An example is given at the end of this page. Of course we need the relativTo ThingID^? back in the PoseData for that. -- MartinBauer - 04 Nov 2003

module DWARF {
   enum SceneCommand { MoveObject, CreateObject, ReplaceObject, DeleteObject, ReplaceScene, SuperImpose, SetProperty };
   struct SceneAction {
      SceneCommand command;   // mandatory
      ThingID objectId;      // mandatory
      ThingID relativeTo;   // for MoveObject, CreateObject
      String vrml;         // for CreateObject/ReplaceObject/ReplaceScene/Superimpose
      String key;         // for SetProperty
      String value;      // for SetProperty 
   };
};

This shall be the meaning of the commands:

• CreateObject: Add an 3D Object with given geometry to the SceneGraph
• MoveObject: Move an Object from one place in the scenegraph to another
• ReplaceObject: Replace an Object in the scenegraph by another representation
• DeleteObject: Delete a 3D Object from the SceneGraph

• ReplaceScene: Replace the whole 3D Scene
• SuperImpose: Overlay the 3D Scene with 2D Information e.g. Text

Additionally, it would be nice to have a posiibility to change arbitrary parameters.

• SetProperty: change arbitrary parameters - need to identify the shape node and the (key,value) pair

This should be very easy since The SoAdvancedShapeKit class that we use to encapsulate geometry provides a setProperty(key,value) method. -- OtmarHilliges - 01 Nov 2003

Events vs. Method Calls

It may be arguable that for scene graph modifications method calls would be more appropriate then events. Any Pros/Cons? -- MartinBauer - 04 Nov 2003

• Pro: You know when you are connected -- MartinBauer - 04 Nov 2003
• Con: This assumes 1 to 1 connections -- MartinBauer - 04 Nov 2003
• Pro: More flexibility in the commands. Right now we have a rather weak implementation of the Command Pattern; maybe we should simply design the whole event as a proper command pattern?

(by the way, this is an issue. you may add subissues and subissues up to infinity here ;-)

Restructuring the ServiceDescription

The viewer should adapt itself dynamically to the incoming connections. that is why it is not useful to configure the number of threads using the configuration XML file. Instead, the viewer should have few almost universal interfaces that adapt to the incoming connections. This is where i will add some UML diagrams probably.

The viewer should by default have the following needs and abilities:

	Name	Type	Protocol			attributes/predicates
Need	viewpoint	PoseData	SvcProtPushConsumer	0	1	(ThingType?=viewpoint)	this is the tracked viewpoint of the viewer
Need	calibration	SvcProtObjrefImporter?		0	1		this is the interface to set a calibration matrix
Need	objects	PoseData	SvcProtPushConsumer, SvcSession	0	∞		tracked positions of objects in the scene
Need	scene	SceneData*	SvcProtPushConsumer, SvcSession	0	∞		changes in the scene
Ability	status	Service					status inteface, describing the number and names of the connected services

* this refers to the "new" SceneData described in this document.

• the Need for the viewpoint has the predicate "(ThingType^?=viewpoint)" set by default. if you want to provide a viewpoint, simply set ypur attribute accordingly. There can be at most one viewpoint at a time.
• the Need for the calibration matrix should probably be a SvcProtObjrefImporter^? since this allows to easily detect when the services are connected and then set the calibration matrix (intrinsic camera parameters) accordingly. There needs to be a reasonable calibration matrix set by default. The extrinsic camera parameters are set using the viewpoint Need.
• the Need for tracked objects has to use the SvcSession interface. For each new connection, a new object is used to consume the events. The attributes for ThingID and Chair.ThingType should match the ThingID and Chair.ThingType field inside the PoseData events. Hence the searching for a particular node in the scenegraph can be done in the newSession callback, returning a valid session only when the required object is in fact found in the scenegraph. Asa: Is there a way to end a session when the corresponding object gets removed from the scenegraph?
  • another option would be: for every named object in the scenegraph, i.e. an object that has a ThingID^? associated, a new Need is created for the PoseData of that object. If somebody wants to provide that PoseData, then its used after the ServiceManager creates the connection. This sounds cool.
  • then each of these abilities does not need to use the SvcSession interface since it works with a single object in the scenegraph only. there remains the question what happens with multiple trackers providing tracking data for that object. could get interesting in the UbiquitousTracking^? scenario. for the time beeing assume that there is no probelem here.
• the Need for scene manipulation needs to implement the session interface as well. Or in fact, it should. I have no particular reason for that right now but i will make one up soon Usually, events on this Need should not be that frequent.

Modifying Objects inside a large Scenegraph

The MoveObject command (and, if exists, the ChangeProperty command) needs to be able to modify any node inside the scenegraph and not only nodes that have been aded by the CreateObject method. Since sometimes large model files are used that are not possible to send over the network, they need to be loaded initially using the command line argument when starting the viewer. sometimes we still want to modify parts of that scene. One Example: We load one large CT scan of a patient. this three-dimensional reconstruction is seperated into the bones, the skin and various organs. now i want to set the visibility attribute of each of these different parts of the model to on or off. This may be done by adding a new node to the scenegraph containing the ThingID^?. If you know of any better solution, write it down here.

Real World Example

Some call it use cases. Some may call it scenarios. I dont care.

Sheep

This is what happens in the new Sheep application.

• The viewer is started with the landscape as default model.
• A python script providees the "application logic" for the sheep game. This consist of basically two Abilities: setting the calibration matrix according to the optical properties of the ceiling mounted beamer and setting the viewpoint to the pose of the beamer.
• A Sheep starts up. The sheep gets connected to the objects and to the scene outlets of the Viewer: The sheep adds itself to the scenegraph, and now the viewer creates a new need for the posedata of that sheep. the service manager creates that connection and the sheep begings to graze happily.
• when the sheep changes its color for some reason, it uses the ReplaceObejct command, or possibly the ChangeProperty command.
  • when the sheep dies, it uses the DeleteScene command to remove itself safely from the scene.
  • when a sheep process is killed, the viewer needs to take care to remove that now unnedded sheep from the scene (good). If this is not desirable, we could use another service that registers to all the sheep and all the sheeps DeleteScene commands and, when a sheeps connection gets cleaned up by the service manager which has not yet removed itself from the scene, that service does the garbage collection (bad).
• Picking up Sheep: This is similar to the wall moving in Archie. When a Sheep gets picked up, the Sheep itself has to handle all that by itself: It receives an Event that it was picked up by some object, identified by its ThingID^?, the Sheep send a MoveObject event to the viewers setting its position to some position realtive to that object. It then continues to do nothing, sending out constant PoseData events, or even walking around in tiny circles on the hand until it receves the event that is has been dropped again. then it sends another MoveObject command setting the relative ThingID^? back to the original (probably the landscape).

Q: We might want to think about speed problems with this approach. It could take some time to create/connect and disconnect communication channels like this (Is this correct Asa?). So that we have a lot of sheep standing around not beeing connected to PoseData.

HEART

This is how the ApplicationHeart would use the proposed redesign of the viewer (w/ or w/o video background).

• The viewer is started with the CT model as startup model.
• a python script provides the application logic, adding the needed robot arms to the scene and setting the camera parameters.
• the camera pose is tracked and the resulting PoseData gets connected to the viewpoint need of the viewer.
• once the robot arms are added to the scene, the Viewer develops the needs for the robot arms pose, which is sent by the tracking (and calibration) subsystem. the ServiceManager then connects everything.
• SceneData events using the ChangeProperty command are used to show and hide individual organs

Fairy Tail

For all who do not know what the ServiceFairyTail^? is (besides beiing not documented yet): Basically, this is the well-known breadcrumbs application where some object in the scene leaves a tail of points whe it is moving around (similar to that funny mouse pointer back in the days of windows 3.1). This can be used to have a magic wand leave a trace of stars when moving, or to have virtual breadcrumbs on the floor when people moved by recently (this was published already somewhere?).

i still dont know how to model that here. Main problem: For performance reasons its desirable to send out all PoseData for all breadcrumbs using a single ability...

Headlight

If there is a light source in the scene, the headlight should be turned off. (ChristianSandor)