MaPoseEstimationYanDI

Chair for Computer Aided Medical Procedures & Augmented Reality
Lehrstuhl für Informatikanwendungen in der Medizin & Augmented Reality

THIS WEBPAGE IS DEPRECATED - please visit our new website

Memory-enhanced Categrory-Level Pose Estimation

Abstract

Category-level pose estimation jointly estimates the 6D pose: Rotation and translation, and object size for unseen objects with known category labels. Currently, the SOTA methods in 9D are FS-Net [1] and DualPoseNet? [2]. And one straightforward idea to improve the performance is to introduce priors into the network. ShapePrior? [3] and CPS [4] leverage the point cloud to represent the mean shape of each category. FS-Net adopts the average size of each category. We, instead, can use a memory module to store typical shapes of each category, similar to point cloud segmentation methods [5].

The way to establish the memory module: 1 First we train the network to extract features and then utilize the feature to reconstruct observed points, as in FS-Net. 2 Assume the features follow GM distribution, we can use a K-means to build the module, or some other unsupervised learning methods may be doable.

The way to train the network: Our network structure is similar to FS-Net and ShapePrior?, thus the training procedures may be similar too.

References:

[1] Chen, W., Jia, X., Chang, H. J., Duan, J., Shen, L., & Leonardis, A. (2021). FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1581-1590).

[2] Lin, J., Wei, Z., Li, Z., Xu, S., Jia, K., & Li, Y. (2021). DualPoseNet?: Category-level 6D Object Pose and Size Estimation using Dual Pose Network with Refined Learning of Pose Consistency. arXiv preprint arXiv:2103.06526.

[3] Tian, M., Ang, M. H., & Lee, G. H. (2020, August). Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation. In European Conference on Computer Vision (pp. 530-546). Springer, Cham.

[4] Manhardt, F., Wang, G., Busam, B., Nickel, M., Meier, S., Minciullo, L., ... & Navab, N. (2020). CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning. arXiv preprint arXiv:2003.05848.

[5] He, T., Gong, D., Tian, Z., & Shen, C. (2020). Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation. arXiv preprint arXiv:2001.01349

Resources

Literature

If you are interested in any of these topics, please contact us via e-mail.

Yan Di

Federico Tombari

Students.ProjectForm
Title: Memory-enhanced Categrory-Level Pose Estimation
Abstract: Category-level pose estimation jointly estimates the 6D pose: Rotation and translation, and object size for unseen objects with known category labels. Currently, the SOTA methods in 9D are FS-Net [1] and DualPoseNet? [2]. And one straightforward idea to improve the performance is to introduce priors into the network. ShapePrior? [3] and CPS [4] leverage the point cloud to represent the mean shape of each category. FS-Net adopts the average size of each category. We, instead, can use a memory module to store typical shapes of each category, similar to point cloud segmentation methods [5]. The way to establish the memory module: 1 First we train the network to extract features and then utilize the feature to reconstruct observed points, as in FS-Net. 2 Assume the features follow GM distribution, we can use a K-means to build the module, or some other unsupervised learning methods may be doable. The way to train the network: Our network structure is similar to FS-Net and ShapePrior?, thus the training procedures may be similar too. References: [1] Chen, W., Jia, X., Chang, H. J., Duan, J., Shen, L., & Leonardis, A. (2021). FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1581-1590). [2] Lin, J., Wei, Z., Li, Z., Xu, S., Jia, K., & Li, Y. (2021). DualPoseNet?: Category-level 6D Object Pose and Size Estimation using Dual Pose Network with Refined Learning of Pose Consistency. arXiv preprint arXiv:2103.06526. [3] Tian, M., Ang, M. H., & Lee, G. H. (2020, August). Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation. In European Conference on Computer Vision (pp. 530-546). Springer, Cham. [4] Manhardt, F., Wang, G., Busam, B., Nickel, M., Meier, S., Minciullo, L., ... & Navab, N. (2020). CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning. arXiv preprint arXiv:2003.05848. [5] He, T., Gong, D., Tian, Z., & Shen, C. (2020). Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation. arXiv preprint arXiv:2001.01349
Student:  
Director: Federico Tombari
Supervisor: Yan DI Yanyan Li
Type: Master Thesis
Area: Computer Vision
Status: finished
Start:  
Finish:  
Thesis (optional):  
Picture:  


Edit | Attach | Refresh | Diffs | More | Revision r1.2 - 22 Jul 2021 - 12:26 - YanyanLi