Evaluating human skills using deep neural networks
Supervision: Prof. Dr. Nassir Navab
, Dr. Seong Tae Kim
, Tobias Czempiel
Recently, deep learning has great success in various applications such as image recognition, object detection, and medical applications, etc. Using YouTube?
and Vimeo, how-to videos are widely used to transfer the skills of experts. It is done by capturing the reference video for a specific task and users could learn a new skill according to the how-to videos. Then, how to evaluate the newly learned skills? Usually, it has to be evaluated by the experts but it takes high cost. To address this problem, evaluating human skills with the video is required [1-4]. Human skill evaluation or determination is a research area where researchers develop a new solution to automatically assess the human skills from the video. This technology could be extended to surgical skill assessment in medical applications where it’s accuracy becomes much more important [5-6]. In this project, we will develop a solution to automatically assess the human skills from the video.
- Good understanding of statistics and machine learning methods.
- Very good programming skills in Python & TensorFlow? / PyTorch?
- IFL Lab (http://campar.in.tum.de/Chair/IFL). A student would be assigned a seat at IFL lab and the weekly-based regular meeting will be scheduled at IFL or Garching.
 Doughty, Hazel, Dima Damen, and Walterio Mayol-Cuevas. "Who's Better? Who's Best? Pairwise Deep Ranking for Skill Determination." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6057-6066. 2018.
 Parmar, Paritosh, and Brendan Tran Morris. "What and how well you performed? A multitask learning approach to action quality assessment." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 304-313. 2019.
 Doughty, Hazel, Walterio Mayol-Cuevas, and Dima Damen. "The Pros and Cons: Rank-aware temporal attention for skill determination in long videos." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7862-7871. 2019.
 Li, Y. Huang, M. Cai, and Y. Sato, "Manipulation-skill assessment from videos with spatial attention network," International Conference on Computer Vision Workshop (ICCVW), 2019.
 Zia, Aneeq, Yachna Sharma, Vinay Bettadapura, Eric L. Sarin, Mark A. Clements, and Irfan Essa. "Automated assessment of surgical skills using frequency analysis." In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 430-438. Springer, Cham, 2015.
 Zia, Aneeq, Yachna Sharma, Vinay Bettadapura, Eric L. Sarin, and Irfan Essa. "Video and accelerometer-based motion analysis for automated surgical skills assessment." International journal of computer assisted radiology and surgery 13, no. 3 (2018): 443-455.
If you are interested, please contact us via e-mail: email@example.com