University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Semantically Coherent 4D Scene Flow of Dynamic Scenes

Mustafa, Armin and Hilton, Adrian (2019) Semantically Coherent 4D Scene Flow of Dynamic Scenes International Journal of Computer Vision.

Semantically Coherent 4D Scene Flow of Dynamic Scenes.pdf - Version of Record
Available under License Creative Commons Attribution.

Download (6MB) | Preview


Simultaneous semantically coherent object-based long-term 4D scene flow estimation, co-segmentation and reconstruction is proposed exploiting the coherence in semantic class labels both spatially, between views at a single time instant, and temporally, between widely spaced time instants of dynamic objects with similar shape and appearance. In this paper we propose a framework for spatially and temporally coherent semantic 4D scene flow of general dynamic scenes from multiple view videos captured with a network of static or moving cameras. Semantic coherence results in improved 4D scene flow estimation, segmentation and reconstruction for complex dynamic scenes. Semantic tracklets are introduced to robustly initialize the scene flow in the joint estimation and enforce temporal coherence in 4D flow, semantic labelling and reconstruction between widely spaced instances of dynamic objects. Tracklets of dynamic objects enable unsupervised learning of long-term flow, appearance and shape priors that are exploited in semantically coherent 4D scene flow estimation, co-segmentation and reconstruction. Comprehensive performance evaluation against state-of-the-art techniques on challenging indoor and outdoor sequences with hand-held moving cameras shows improved accuracy in 4D scene flow, segmentation, temporally coherent semantic labelling, and reconstruction of dynamic scenes.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors : Mustafa, Armin and Hilton, Adrian
Date : 3 October 2019
Funders : Royal Academy of Engineering Research Fellowship, Engineering and Physical Sciences Research Council (EPSRC)
DOI : 10.1007/s11263-019-01241-w
Copyright Disclaimer : © The Author(s) 2019. Open Access. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Uncontrolled Keywords : Semantic 4D flow; Reconstruction; Segmentation
Depositing User : Clive Harris
Date Deposited : 04 Nov 2019 09:17
Last Modified : 04 Nov 2019 09:17

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800