Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

THIS WEBPAGE IS DEPRECATED - please visit our new website

Investigating 3D Capsule Networks


In recent years, convolutional neural networks (CNN) have been a key component in the progress of computer vision by means of weight sharing and translational invariance. However, there are two main downsides to CNNs: sub-sampling operation loses the precise spatial relationships, and CNNs cannot extrapolate their understanding of geometric relationships to entirely new viewpoints [1]. Sabour et al. proposed a novel neural network called capsule networks in 2017 in [2] to address these concerns. The capsule networks have achieved higher accuracy than traditional convolutional models with a similar number of parameters. Since its introduction, capsule networks have been used in various tasks in computer vision including shape alignment [3], video object segmentation [4], and classi cation of breast cancer [5].

In this research, we will explore two areas of improvement of capsule networks. Firstly, the interpretability of the existing capsule network architectures usually lack the ability to explain what each capsule corresponds to. Therefore, we will examine the latent capsule in the existing capsule networks to understand their meaning. Secondly, we will reconstruct 3D shapes from rendered images taken from different viewpoints. This path potentially includes implementing a previous work as the source code of this work has not been shared publicly.

3D Capsule Networks


[1] Geoffrey Hinton. Taking inverse graphics seriously.

[2] Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In Neurips, 2017.

[3] Yongheng Zhao, Tolga Birdal, Jan Eric Lenssen, Emanuele Menegatti, Leonidas Guibas, and Federico Tombari. Quaternion equivariant capsule networks for 3d point clouds. arXiv preprint arXiv:1912.12098, 2019.

[4] Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. Capsulevos: Semi-supervised video object segmentation using capsule routing. In ICCV 2019.

[5] Tomas Iesmantas and Robertas Alzbutas. Convolutional capsule network for classifi cation of breast cancer histology images. In International Conference Image Analysis and Recognition, 2018.

[6] Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 3d point capsule networks. In ICCV, 2019.

Title: Investigating 3D Capsule Networks
Student: Dilara Gökay
Director: Federico Tombari
Supervisor: Evin Pınar Örnek
Type: DA/MA/BA
Area: Computer Vision
Status: running
Start: 10.2020
Finish: 03.2021
Thesis (optional):  

Edit | Attach | Refresh | Diffs | More | Revision r1.1 - 01 Nov 2020 - 16:30 - EvinPinarOrnek