University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Efficient 3D Morphable Face Model Fitting

Hu, G, Yan, Fei, Kittler, Josef, Christmas, William, Chan, Chi Ho, Feng, Zhenhua and Huber, Patrik (2017) Efficient 3D Morphable Face Model Fitting Pattern Recognition, 67. pp. 366-379.

[img] Text
2017_PR.pdf - Accepted version Manuscript
Restricted to Repository staff only until 22 February 2018.
Available under License : See the attached licence file.

Download (2MB)
[img]
Preview
Text (licence)
SRI_deposit_agreement.pdf
Available under License : See the attached licence file.

Download (33kB) | Preview

Abstract

3D face reconstruction of shape and skin texture from a single 2D image can be performed using a 3D Morphable Model (3DMM) in an analysis-by-synthesis approach. However, performing this reconstruction (fitting) efficiently and accurately in a general imaging scenario is a challenge. Such a scenario would involve a perspective camera to describe the geometric projection from 3D to 2D, and the Phong model to characterise illumination. Under these imaging assumptions the reconstruction problem is nonlinear and, consequently, computationally very demanding. In this work, we present an efficient stepwise 3DMM-to-2D image-fitting procedure, which sequentially optimises the pose, shape, light direction, light strength and skin texture parameters in separate steps. By linearising each step of the fitting process we derive closed-form solutions for the recovery of the respective parameters, leading to efficient fitting. The proposed optimisation process involves all the pixels of the input image, rather than randomly selected subsets, which enhances the accuracy of the fitting. It is referred to as Efficient Stepwise Optimisation (ESO). The proposed fitting strategy is evaluated using reconstruction error as a performance measure. In addition, we demonstrate its merits in the context of a 3D-assisted 2D face recognition system which detects landmarks automatically and extracts both holistic and local features using a 3DMM. This contrasts with most other methods which only report results that use manual face landmarking to initialise the fitting. Our method is tested on the public CMU-PIE and Multi-PIE face databases, as well as one internal database. The experimental results show that the face reconstruction using ESO is significantly faster, and its accuracy is at least as good as that achieved by the existing 3DMM fitting algorithms. A face recognition system integrating ESO to provide a pose and illumination invariant solution compares favourably with other state-of-the-art methods. In particular, it outperforms deep learning methods when tested on the Multi-PIE database.

Item Type: Article
Subjects : Electronic Engineering
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
NameEmailORCID
Hu, GUNSPECIFIEDUNSPECIFIED
Yan, FeiF.Yan@surrey.ac.ukUNSPECIFIED
Kittler, JosefJ.Kittler@surrey.ac.ukUNSPECIFIED
Christmas, WilliamW.Christmas@surrey.ac.ukUNSPECIFIED
Chan, Chi HoChiho.Chan@surrey.ac.ukUNSPECIFIED
Feng, Zhenhuaz.feng@surrey.ac.ukUNSPECIFIED
Huber, Patrikp.huber@surrey.ac.ukUNSPECIFIED
Date : 22 February 2017
Identification Number : 10.1016/j.patcog.2017.02.007
Copyright Disclaimer : © 2017. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/
Uncontrolled Keywords : face recognition; face reconstruction; 3D Morphable Model
Related URLs :
Depositing User : Symplectic Elements
Date Deposited : 24 Feb 2017 14:43
Last Modified : 31 Oct 2017 19:09
URI: http://epubs.surrey.ac.uk/id/eprint/813634

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year


Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800