University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Real-Time Multi-person Motion Capture from Multi-view Video and IMUs.

Malleson, Charles, Collomosse, John and Hilton, Adrian (2019) Real-Time Multi-person Motion Capture from Multi-view Video and IMUs. International Journal of Computer Vision.

Full text not available from this repository.

Abstract

A real-time motion capture system is presented which uses input from multiple standard video cameras and inertial measurement units (IMUs). The system is able to track multiple people simultaneously and requires no optical markers, specialized infra-red cameras or foreground/background segmentation, making it applicable to general indoor and outdoor scenarios with dynamic backgrounds and lighting. To overcome limitations of prior video or IMU-only approaches, we propose to use flexible combinations of multiple-view, calibrated video and IMU input along with a pose prior in an online optimization-based framework, which allows the full 6-DoF motion to be recovered including axial rotation of limbs and drift-free global position. A method for sorting and assigning raw input 2D keypoint detections into corresponding subjects is presented which facilitates multi-person tracking and rejection of any bystanders in the scene. The approach is evaluated on data from several indoor and outdoor capture environments with one or more subjects and the trade-off between input sparsity and tracking performance is discussed. State-of-the-art pose estimation performance is obtained on the Total Capture (mutli-view video and IMU) and Human 3.6M (multi-view video) datasets. Finally, a live demonstrator for the approach is presented showing real-time capture, solving and character animation using a light-weight, commodity hardware setup.

Item Type: Article
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering > Centre for Vision Speech and Signal Processing
Authors :
NameEmailORCID
Malleson, Charlescharles.malleson@surrey.ac.uk
Collomosse, JohnJ.Collomosse@surrey.ac.uk
Hilton, AdrianA.Hilton@surrey.ac.uk
Date : 17 December 2019
Funders : Innovate UK, European Union Horizon 2020
DOI : 10.1007/s11263-019-01270-5
Copyright Disclaimer : © The Author(s) 2019. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Uncontrolled Keywords : Pose estimation; Motion capture; IMU; Multi-view video; Real-time; Multi-person
Depositing User : Diane Maxfield
Date Deposited : 24 Jan 2020 15:25
Last Modified : 05 Feb 2020 10:38
URI: http://epubs.surrey.ac.uk/id/eprint/853403

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800