Thesis or Guided Research in Weakly-Supervised Action Segmentation
Abstract
Activity understanding in videos became a popular research topic in the computer vision community because of its application to video analysis. Thanks to large-scale labeled video datasets[1,2], classifying activities in trimmed videos made significant progress in recent years. In contrast, action segmentation which requires finding boundaries and action labels in untrimmed videos is still a challenging problem. The key issue is untrimmed videos are usually quite long and contain multiple sub-activities, therefore gathering a large scale video dataset for action segmentation is time-consuming and cumbersome. To address these issues, recent research on this direction focuses on training deep architectures with weak labels. This project will focus on designing a new method for action segmentation with weak labels. The proposed approach will be compared against SOTA methods [3,4,5] on publicly available action segmentation datasets.PS: This project is going to be remotely supervised due to Covid-19
Sample activity segmentation:
Literature
[1] Smaira, Lucas, et al. "A Short Note on the Kinetics-700-2020 Human Action Dataset." arXiv preprint arXiv:2010.10864 (2020). [2] Karpathy, Andrej, et al. "Large-scale video classification with convolutional neural networks." Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2014. [3] Li, Jun, Peng Lei, and Sinisa Todorovic. "Weakly supervised energy-based learning for action segmentation." Proceedings of the IEEE International Conference on Computer Vision. 2019. [4] Chang, Chien-Yi, et al. "D3tw: Discriminative differentiable dynamic time warping for weakly supervised action alignment and segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [5] Ding, Li, and Chenliang Xu. "Weakly-supervised action segmentation with iterative soft boundary assignment." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.Requirements
Strong programming skills with Python Knowledge with PyTorch? or TensorFlow? General Deep Learning knowledge. Research Experience.Application
If you are interested, please contact us via e-mail: Huseyin Coskun Federico Tombari| Students.ProjectForm | |
|---|---|
| Title: | Weakly-Supervised Action Segmentation |
| Abstract: | Activity understanding in videos became a popular research topic in the computer vision community because of its application to video analysis. Thanks to large-scale labeled video datasets[1,2], classifying activities in trimmed videos made significant progress in recent years. In contrast, action segmentation, which requires finding boundaries and action labels in untrimmed videos, is still a challenging problem. The key issue is untrimmed videos are usually quite long and contain multiple sub-activities, therefore gathering a large scale video dataset for action segmentation is time-consuming and cumbersome. To address these issues, recent research on this direction focuses on training deep architectures with weak labels. This project will focus on designing a new method for action segmentation with weak labels. The proposed approach will be compared against new SOTA methods [3,4,5] on action segmentation datasets. |
| Student: | |
| Director: | Federico Tombari |
| Supervisor: | Huseyin Coskun, Prof. Nassir Navab |
| Type: | DA/MA/BA |
| Area: | Machine Learning, Computer Vision |
| Status: | open |
| Start: | |
| Finish: | |
| Thesis (optional): | |
| Picture: | |