3D Scene Understanding Leveraging Scene Graphs

For a collaboration with researchers from Google, we are currently looking for a very motivated student interested in a master’s thesis project or guided research. The project involves topics including deep learning, 3D computer vision, transformer networks [1], and scene graphs [2]. Scene graphs are a compact data representation that describes an image or 3D model of a scene. Each node of this graph represents an object, while the edges represent relationships/interactions between these objects, e.g. "chair - standing on - floor" or "guitar - leaning against - cupboard". An example dataset can be found here [3].

The aim of the project is to explore novel architectures for scene graph manipulation based on current research trends towards improving the representation capabilities of scene graphs and address known issues in the representation of scenes.

Prerequisites: The candidate should have interest and knowledge in deep learning and computer vision, be comfortable with Python and preferably have some experience with a deep learning framework, such as PyTorch^? or TensorFlow^?.

Supervision: Helisa Dhamo, Fabian Manhardt, PD Dr. Federico Tombari

Please drop an email to Helisa Dhamo or Fabian Manhardt to find out more.

[1] “Attention Is All You Need.” A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin. NeurIPS^? 2017. https://arxiv.org/abs/1706.03762
[2] “Image Generation from Scene Graphs.” J. Johnson, A. Gupta, L. Fei-Fei. CVPR 2018. https://arxiv.org/abs/1804.01622
[3] “Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions.” J. Wald, H. Dhamo, N. Navab, F. Tombari. CVPR 2020. https://arxiv.org/abs/2004.03967