MaWeakActSeg

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

Thesis or Guided Research in Weakly-Supervised Action Segmentation

Abstract

Activity understanding in videos became a popular research topic in the computer vision community because of its application to video analysis. Thanks to large-scale labeled video datasets[1,2], classifying activities in trimmed videos made significant progress in recent years. In contrast, action segmentation which requires finding boundaries and action labels in untrimmed videos is still a challenging problem. The key issue is untrimmed videos are usually quite long and contain multiple sub-activities, therefore gathering a large scale video dataset for action segmentation is time-consuming and cumbersome. To address these issues, recent research on this direction focuses on training deep architectures with weak labels. This project will focus on designing a new method for action segmentation with weak labels. The proposed approach will be compared against SOTA methods [3,4,5] on publicly available action segmentation datasets.


PS: This project is going to be remotely supervised due to Covid-19

Sample activity segmentation:

Literature

[1] Smaira, Lucas, et al. "A Short Note on the Kinetics-700-2020 Human Action Dataset." arXiv preprint arXiv:2010.10864 (2020).
[2] Karpathy, Andrej, et al. "Large-scale video classification with convolutional neural networks." Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2014.
[3] Li, Jun, Peng Lei, and Sinisa Todorovic. "Weakly supervised energy-based learning for action segmentation." Proceedings of the IEEE International Conference on Computer Vision. 2019.
[4] Chang, Chien-Yi, et al. "D3tw: Discriminative differentiable dynamic time warping for weakly supervised action alignment and segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
[5] Ding, Li, and Chenliang Xu. "Weakly-supervised action segmentation with iterative soft boundary assignment." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

Requirements

Strong programming skills with Python
Knowledge with PyTorch? or TensorFlow?
General Deep Learning knowledge.
Research Experience.

Application

If you are interested, please contact us via e-mail:

Huseyin Coskun

Federico Tombari

Students.ProjectForm
Title: Weakly-Supervised Action Segmentation
Abstract: Activity understanding in videos became a popular research topic in the computer vision community because of its application to video analysis. Thanks to large-scale labeled video datasets[1,2], classifying activities in trimmed videos made significant progress in recent years. In contrast, action segmentation, which requires finding boundaries and action labels in untrimmed videos, is still a challenging problem. The key issue is untrimmed videos are usually quite long and contain multiple sub-activities, therefore gathering a large scale video dataset for action segmentation is time-consuming and cumbersome. To address these issues, recent research on this direction focuses on training deep architectures with weak labels. This project will focus on designing a new method for action segmentation with weak labels. The proposed approach will be compared against new SOTA methods [3,4,5] on action segmentation datasets.
Student:  
Director: Federico Tombari
Supervisor: Huseyin Coskun, Prof. Nassir Navab
Type: DA/MA/BA
Area: Machine Learning, Computer Vision
Status: open
Start:  
Finish:  
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.1 - 13 Nov 2020 - 22:20 - HuseyinCoskun