Exploring 3D Scene Graphs for Surgical Operating Rooms

Abstract

Scene graphs are structures, which can be used to describe an image or a 3D environment in a semantically rich and compact manner. Nodes of these graphs represent objects, whereas the connections between the nodes represent relationships.

In our work, we plan to utilize 3D scene graphs to describe medical operations (e.g. surgeries). The goal is to investigate 3D temporal scene graphs along with the dataset requirements, and then explore the use cases of these graphs in OR workflow, such as detecting the current phase of the surgery, identifying possible anomalies or predicting roles seen in the operation.

Literature

Johanna Wald, Helisa Dhamo, Nassir Navab, Federico Tombari, "Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions" CVPR 2020.

Antoni Rosinol, Arjun Gupta, Marcus Abate, Jingnan Shi, Luca Carlone, "3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans" Comput. Graph. Forum Journal 2020.

V. Srivastav, T. Issenhuth, A. Kadkhodamohammadi, M. de Mathelin, A. Gangi, N. Padoy, MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimation, MICCAI-LABELS, arXiv preprint, 2018.