6 July 2018, MI 03.13.010 Invited Talk by Prof. William (Sandy) Wells Segmentation is a fundamental task for extracting semantically meaningful regions from an image. The goal of segmentation algorithms is to accurately assign object labels to each image location. However, image-noise, shortcomings of algorithms, and image ambiguities cause uncertainty in label assignment. Estimating the uncertainty in label assignment is important in multiple application domains, such as segmenting tumors from medical images for radiation treatment planning. One way to estimate these uncertainties is through the computation of posteriors of Bayesian models, which is computationally prohibitive for many practical applications. On the other hand, most computationally efficient methods fail to estimate label uncertainty. [...]
6 December 2018, 00.12.019 PhD Defense by Christian Rupprecht Nearly all real-world image understanding problems in computer vision are inherently ambiguous. Often, predictive systems do not model this ambiguity and do not consider the possibility that there can be more than just a single outcome for a given problem. This leads to sub-par performance on ambiguous tasks as the model has to account for all possibilities with one answer. We define three typical sources of confusion that render tasks not optimally solvable with a single unique prediction. In this dissertation we describe two principled and general approaches of dealing with ambiguity. First, we elaborate on a method that allows the algorithm to predict multiple instead of one single answer. This is a pragmatic way of dealing with ambiguity: instead of deciding for an exclusive outcome for a given problem, we enable the system to list several possibilities. The second part describes an alternative way to deal with uncertain predictions. Often human perception can provide additional information about a task or application that an intelligent system might have not recognized. Building on the paradigm of human-machine interaction, we show how enabling interaction between the system and a user can improve predictions on the example of semantic segmentation. We describe a novel guiding mechanism that can be seamlessly integrated into the system and shows great potential beyond the demonstrated tasks for several further applications.
19 November 2018, 01.09.014 PhD Defense by Nicola Rieke Visual tracking of surgical instruments is a key component of various computer-assisted interventions, yet a very challenging problem in the field of Computer Vision. This dissertation presents novel approaches which leverage machine learning techniques for precise real-time tracking and 2D pose estimation of instruments. The achieved results demonstrate that the proposed methods based on Random Forests and Deep Learning provide remarkable advantages with respect to the state of the art in terms of accuracy, robustness and generalization.
8 October 2018, 01.06.020 Signed Distance Fields for Rigid and Deformable 3D Reconstruction Capturing three-dimensional environments is a key task in the growing fields of virtual and augmented reality. This thesis addresses the task of 3D reconstruction of both static and dynamic objects and scenes scanned with a single hand-held RGB-D camera, without any markers or prior knowledge. Reconstructing rigid environments requires estimating the six degrees-of-freedom camera pose at every time instance, and subsequently fusing the acquired data into a geometrically consistent computer model. The task of reconstructing deformable objects is more challenging, as additionally the non-rigid motion that occurred in every frame has to be determined and factored out. We propose to tackle both the rigid and deformable reconstruction problems via implicit-to-implicit alignment of SDF pairs without correspondence search. In the static case, we obtain more accurate pose estimates with a framework that permits straightforward incorporation of various additional constraints, such as surface colour and orientation. We start with the reconstruction of small- to medium-scale household objects and demonstrate how to extend the approach to larger spaces such as rooms. To this end, we develop a limited-extent volume strategy that restricts registration to the most geometrically distinctive regions of a scene, leading to significantly improved rotational motion estimation. Finally, we adapt our approach to dynamic scenes by modifying our implicit-to-implicit approach so that new data is incremented appropriately. For this purpose we evolve an initial SDF to a target SDF by imposing rigidity constraints that require the underlying deformation field to be approximately Killing, i.e. volume-preserving and generating locally isometric motions. Alternatively, we employ gradient flow in the smooth Sobolev space, which favours global deformations over finer-scale details. These strategies also circumvent explicit correspondence search and thus avoid the repeated conversion between SDF and mesh representations that other techniques entail. Nevertheless, we ensure that correspondence information can be recovered by proposing two strategies based on Laplacian eigenfunctions, which are known to encode natural deformation patterns. Thanks to the used SDF representation, our non-rigid reconstruction approach is able to handle topological changes and fast motion, which are major obstacles for existing approaches.
10 September 2018, IFL Invited Talk by Dr. Hayato Itoh In colon cancer screening, polyp size estimation using only colonoscopy images or videos is difficult even for expert physicians although the size information of polyps is important for diagnosis. To construct a fully automated computer-aided diagnosis (CAD) pipeline, a robust and precise polyp size estimation method is highly desired. However, the size estimation problem of a three-dimensional object from a single two-dimensional image is ill-posed due to the lack of three-dimensional spatial information. To circumvent this challenge, we formulate a relaxed form of size estimation as a binary-size classification problem and solve it by a new deep neural network architecture. This relaxed form of size estimation is defined as a two-category classification: under and over a certain polyp dimension criterion that would provoke different clinical treatments (resecting the polyp or not). Our proposed deep learning architecture estimates the depth map from an input colonoscopic RGB image using unsupervised deep learning, and integrates RGB with the computed depth information to produce a four-channel RGB-D imagery data, that is subsequently encoded as a pipeline to extract deep RGB-D image features and facilitate the size classification into two categories: under and over 10mm polyps. We collect a large dataset of colonoscopic videos of totally over 16 hours is constructed for the evaluation of our proposed method. Using this dataset, we evaluate the accuracies of both polyp detection and binary polyp-size estimation since detection is a prerequisite step of a fully automated CAD system. The experimental results show that our proposed method achieves 79.2% accuracy for binary polyp-size classification. We also combine the extraction features by our deep learning architecture and classification of short video clips using a long short-term memory (LSTM) network. Polyp detection (if the video clip contains a polyp or not) shows 88.8% sensitivity when employing the spatio-temporal image feature extraction and classification.