RGBD Augmented C-arms provide an overlay of X-ray image over a video image. As an additional information, depth information at every color pixel is also provided. RGBD Augmented C-arms are intended to be used inside the Operating Room for performing surgery, providing an overview of the surgeon actions in context with the X-ray image. In most of the surgeries, the surgeon place its hands in the field of view of the device as it corresponds to the surgical field. Numerous works have lately been published concerning depth-based hand tracking using Machine Learning, from Random Forests [1] to Deep Learning [2]. Tracking the joints of the surgeon´s hands when placed in the field of view of the RGBD augmented C-arm is of numerous interests: gesture control user interface (the surgeon can control directly with gesture the blending value of X-ray), pose estimation, diminished reality (as we can simplify the hand to a centerline representation).
Video from [2]:
Students.ProjectForm | |
---|---|
Title: | Depth-based hand tracking for RGBD Augmented C-arms |
Abstract: | |
Student: | |
Director: | Nassir Navab |
Supervisor: | Séverine Habert |
Type: | DA/MA/BA |
Area: | Machine Learning, Computer-Aided Surgery, Computer Vision, Medical Augmented Reality |
Status: | open |
Start: | |
Finish: | |
Thesis (optional): | |
Picture: |