BA/MA: Organs at Risk Detection and Localization for Radiation Therapy Planning using Transformers.

Advisor: Prof. Bjoern Menze
Supervision by: Fernando Navarro
Suprosanna Shit

Abstract

In this project, we explore 3D transformers for organs at risk (localization) in volumetric medical imaging relevant for radiation therapy planning. The student will apply and develop state-of-the-art transformers for medical image detection and localization.

Figure 1: Overview of the end-to-end object detection with transformers architecture proposed in [5]

In this project, we explore 3D transformers for organs at risk (localization) in volumetric medical imaging relevant for radiation therapy planning. The student will apply and develop state-of-the-art transformers for medical image detection and localization.

Background and Motivation

Detecting OAR is one of the primary tasks prior to radiation therapy planning to maximize the dose to tumor and minimize the radiation in healthy structures. The current workflow requires hours of manual contouring of OAR, which makes the process slow, time-consuming, and user-dependent. Automatic segmentation of OAR has emerged as a potential solution for OAR delineation for radiation therapy [1,2]. Nevertheless, current approaches for OAR tend to fail for small structures and organs where the contrast to surrounding tissues is difficult to distinguish [3]. As a solution, a detection followed by a segmentation step system can improve the performance of OAR delineation. Given the success of transformers in natural language processing [4], as well as in computer vision for object detection [5], in this thesis, we explore and extend transformers architecture for 3D organs at risk detection and localization. In Figure 1. an example of 2D object detection with transformers propose in [5].

Tasks

The aim of this project is to develop and implement 3D transformers for image volumetric medical imaging for the task of OAR detection.

The student task would be to perform the following tasks:

• Develop and implement 3D transformers to automatically detect OAR in CT, MRI scans.
• Evaluate the performance of developed architectures and training strategies.

Prerequisites

The student should have prior knowledge in :

• Image processing
• Experience with deep learning
• Good python programming skills
• Experience with Pytorch

Contact

Fernando Navarro Suprosanna Shit Bjoern Menze

References

[1] Trullo, R., Petitjean, C., Ruan, S., Dubray, B., Nie, D. and Shen, D., 2017, April. Segmentation of organs at risk in thoracic CT images using a sharpmask architecture and conditional random fields. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) (pp. 1003-1006). IEEE.

[2] Liu, Z., Liu, X., Xiao, B., Wang, S., Miao, Z., Sun, Y. and Zhang, F., 2020. Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. Physica Medica, 69, pp.184-191.

[3] Roth, H.R., Lu, L., Farag, A., Shin, H.C., Liu, J., Turkbey, E.B. and Summers, R.M., 2015, October. Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 556-564). Springer, Cham.

[4] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I., 2017. Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).

[5] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A. and Zagoruyko, S., 2020, August. End-to-end object detection with transformers. In European Conference on Computer Vision (pp. 213-229). Springer, Cham.