Master /or IDP thesis : Parallax-free Xray stitching using depth based pose estimation

Advisor: Nassir Navab
Supervision by: Severine Habert, Pascal Fallavollita

Background

Accurate intra-operative determination of mechanical axis alignment, joint orientation and leg length is crucial for the treatment of knee osteoarthritis, correction of deformities, and long bone shaft fractures. X-ray images acquired by mobile C-arms have a narrow field of view and cannot visualize the entire bone structure of a lower leg for example within one single X-ray image. Accurate evaluation of long bone geometry based on separated individual images remains challenging. Thus, surgeons require visualizing a panoramic image comprising of several C-arm images stitched together showing all lower bones of interest.

Wang et al. proposed a parallax-free method for stitching using inverse kinematics to know how to move the C-arm and the table to get a good stitching. This work relies on CamC, a video-augmented C-arm, present in NARVIS at the Klinikum Innenstadt. The video is used for guidance to get the desired C-arm poses necessary for the stitching. For more details about this please look at this pdf. The limitation of this work is the configuration of the C-arm, in down-the-beam with the Xray source under the patient table and the intensifier above the patient. In this work, the markers used for the pose estimation are attached under the table and therefore visible by the video camera placed next to the Xray source.

Moreover, this configuration loses one main advantage of CamC which is to give the overlay of the patient surface with the Xray, which is possible in top-the-beam configuration when the Xray source is over the patient. The use of markers is in that case impossible.

Objective

The goal of this work is then to enable the pose estimation in the top-the-beam configuration to get the overlay of the Xray stitching on the video stitching. Furthermore, CamC is now equipped of a RGBD camera instead of a simple video camera. In addition to the video, depth information is also acquired co-registered with the video. This can be used to ease the pose estimation, but also to make a 3D surface reconstruction on which the panoramic Xray can be overlaid. As an example of pose estimation techniques that could be used for this project is to do ICP registration between depth 3D points clouds created when acquiring a Xray image.

Tasks:

• Task 1: inverse kinematics, get familiar with CamC code, understanding the inverse kinematics code already implemented in the old CamC code style, rewriting this code in new CamC code style
• Task 2: pose estimation using depth and/or video: implement a precise pose estimation method working in top-the-beam position using the depth and/or the video
• Task 3: parallax-free stitching integration of the Task 1 and Task 2 to perform the parallax-free stitching clinical validation

Requirements for the student:

• Proficient with C++, OpenGL^?, OpenCV^?
• Being interested in registration techniques using RGBD data

Contact

• please, feel free to contact Severine Habert if you are interested by this project