University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors

Trumble, Matthew, Gilbert, Andrew, Malleson, Charles, Hilton, Adrian and Collomosse, John (2017) Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors In: 28th British Machine Vision Conference.

[img]
Preview
Text
Total Capture 3D Human Pose Estimation Fusing Video and Inertial Sensors.pdf - Accepted version Manuscript

Download (7MB) | Preview

Abstract

We present an algorithm for fusing multi-viewpoint video (MVV) with inertial measurement unit (IMU) sensor data to accurately estimate 3D human pose. A 3-D convolutional neural network is used to learn a pose embedding from volumetric probabilistic visual hull data (PVH) derived from the MVV frames. We incorporate this model within a dual stream network integrating pose embeddings derived from MVV and a forward kinematic solve of the IMU data. A temporal model (LSTM) is incorporated within both streams prior to their fusion. Hybrid pose inference using these two complementary data sources is shown to resolve ambiguities within each sensor modality, yielding improved accuracy over prior methods. A further contribution of this work is a new hybrid MVV dataset (TotalCapture) comprising video, IMU and a skeletal joint ground truth derived from a commercial motion capture system. The dataset is available online at http://cvssp.org/data/totalcapture/.

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Trumble, Matthewmatthew.trumble@surrey.ac.ukUNSPECIFIED
Gilbert, AndrewA.Gilbert@surrey.ac.ukUNSPECIFIED
Malleson, Charlescharles.malleson@surrey.ac.ukUNSPECIFIED
Hilton, AdrianA.Hilton@surrey.ac.ukUNSPECIFIED
Collomosse, JohnJ.Collomosse@surrey.ac.ukUNSPECIFIED
Date : 7 September 2017
Funders : Engineering and Physical Sciences Research Council (EPSRC)
Grant Title : Total Capture project
Related URLs :
Depositing User : Clive Harris
Date Deposited : 25 Jul 2017 08:42
Last Modified : 25 Jul 2017 08:44
URI: http://epubs.surrey.ac.uk/id/eprint/841740

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800