Guided Attention Segmentation Networks

Supervision: Prof. Dr. Nassir Navab, Dr. Shadi Albarqouni, Hasan Sarhan

Abstract

Supervised learning always gives better results than weakly or unsupervised learning. However, supervised methods are not applicable in tasks where labelled data is sparse. In the medical field we have access to large sets of data, however most of them are un- or weakly labelled. It would also be highly costly to hire radiologists to label the images. Therefor, for automatic segmentation to be truly useful in the medical field, we need methods that can learn mostly from un- and weakly labelled images.

There are many already existing methods of unsupervised segmentation on common objects e.g. ImageNet^?, but few work with highly specific medical images. U-Net [1] is a supervised autoencoder created specifically for medical applications. It uses data augmentation to maximize the information given by the limited labelled data, and has inspired many similar models.

Wang et al. [2, 3] shows how joint-segmentation can be used to unsupervised segment classes assuming we have a set of images all containing the same object. Guided Attention Inference Network (GAIN)[4]. gives us an additive measure which can be added to pre-existing state-of-the-art segmentation methods such as Seed, Expand and Constrain[5] to improve results. GAIN adds one new loss to help guide the network to explore new areas of the image, and another to take advantage of pixel level labels for those images that contain it.

Samuli Laine and Timo Aila shows how Temporal Ensembling[6] can be used in a semi-supervised fashion. We can compute a supervised loss for where a corresponding label exists, but for the images without any label we can compute the squared difference between the current output of the network, and the output from the ensemble of previous outputs. The goal of this project is to build on different state-of-the-art methods for weakly and unsupervised methods to and test it on medical images.

Requirements:

• Good understanding of statistics and machine learning methods.
• Very good programming skills in Python & TensorFlow^? / PyTorch^?

Location:

• Garching

Literature

[1] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI, 2015.

[2] F. Wang, Q. Huang, and L. J. Guibas. “Image co-segmentation via consistent functional maps,” in ICCV, pages 849–856, 2013.

[3] F. Wang, Q. Huang, M. Ovsjanikov, L. Guibas. “Unsupervised multi-class joint image segmentation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.

[4] K. Li, Z. Wu, K.-C. Peng, J. Ernst, and Y. Fu. “Tell me where to look: Guided attention inference network,” in arXiv preprint arXiv:1802.10171, 2017.

[5] A. Kolesnikov and C. H. Lampert, “Seed, expand and constrain: Three principles for weakly-supervised image segmentation,” in ECCV, 2016, pp. 695–711.

[6] S. Laine, T. Aila, “Temporal ensembling for semi-supervised learning,” in ICLR, 2017.

Resultant Paper