University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Human-Centric Scene Understanding from Single View 360 Video

Fowler, Sam, Kim, Hansung and Hilton, Adrian (2018) Human-Centric Scene Understanding from Single View 360 Video In: 2018 International Conference on 3D Vision (3DV), 05-08 Sep 2018, Verona, Italy.

Human-Centric Scene Understanding from Single View 360 Video.pdf - Accepted version Manuscript

Download (8MB) | Preview


In this paper, we propose an approach to indoor scene understanding from observation of people in single view spherical video. As input, our approach takes a centrally located spherical video capture of an indoor scene, estimating the 3D localisation of human actions performed throughout the long term capture. The central contribution of this work is a deep convolutional encoder-decoder network trained on a synthetic dataset to reconstruct regions of affordance from captured human activity. The predicted affordance segmentation is then applied to compose a reconstruction of the complete 3D scene, integrating the affordance segmentation into 3D space. The mapping learnt between human activity and affordance segmentation demonstrates that omnidirectional observation of human activity can be applied to scene understanding tasks such as 3D reconstruction. We show that our approach using only observation of people performs well against previous approaches, allowing reconstruction of occluded regions and labelling of scene affordances.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Date : 2018
Funders : Engineering and Physical Sciences Research Council (EPSRC)
DOI : 10.1109/3DV.2018.00046
Copyright Disclaimer : © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Related URLs :
Depositing User : Clive Harris
Date Deposited : 31 Oct 2018 15:21
Last Modified : 11 Dec 2018 11:24

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800