Multimodal Deep Learning
Student: Anna Zeldin
Supervision: Prof. Dr. Nassir Navab,
Shadi Albarqouni
Abstract
Diagnostic errors can harm patients and undermine public trust, yet many of them are preventable. Radiology is one of the specialities most liable to claims of medical negligence. Failure to perform the precise detection of all abnormalities in an imaging examination and their accurate diagnosis results in misinterpretation of radiologic images and oversight of abnormalities.
Deep convolutional neural networks have led to a series of breakthroughs for image classification. CNNs enable learning data-driven, highly representative, layered hierarchical image features from sufficient training data. In this master thesis, we address the issue of misinterpretation of radiologic images by applying deep convolutional neural networks to computer-aided detection problems. Created predictive models will leverage medical imaging scans, as well as available medical records and temporal information.
Tasks:
- Proposing a methodology to integrate EMR into CNN
- Employing Deep Learning to attack the problem
- Validate the proposed algorithm
Requirements:
- Good understanding of statistics and machine learning methods.
- Very good programming skills in Python/Matlab
- Willingness to closely collaborate with physicians and experts in machine learning.
Location:
Literature
[1] Choi, E., Bahadori, M.T. and Sun, J., 2015. Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. arXiv preprint arXiv:1511.05942.
[2] Xu, T., Zhang, H., Huang, X., Zhang, S. and Metaxas, D.N., 2016, October. Multimodal Deep Learning for Cervical Dysplasia Diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 115-123). Springer International Publishing.
Resultant Paper