University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Efficient 3D Morphable Face Model Fitting

Hu, G, Yan, Fei, Kittler, Josef, Christmas, William, Chan, Chi Ho, Feng, Zhenhua and Huber, Patrik (2017) Efficient 3D Morphable Face Model Fitting Pattern Recognition, 67. pp. 366-379.

2017_PR.pdf - Accepted version Manuscript
Available under License : See the attached licence file.

Download (2MB) | Preview
Text (licence)
Available under License : See the attached licence file.

Download (33kB) | Preview


3D face reconstruction of shape and skin texture from a single 2D image can be performed using a 3D Morphable Model (3DMM) in an analysis-by-synthesis approach. However, performing this reconstruction (fitting) efficiently and accurately in a general imaging scenario is a challenge. Such a scenario would involve a perspective camera to describe the geometric projection from 3D to 2D, and the Phong model to characterise illumination. Under these imaging assumptions the reconstruction problem is nonlinear and, consequently, computationally very demanding. In this work, we present an efficient stepwise 3DMM-to-2D image-fitting procedure, which sequentially optimises the pose, shape, light direction, light strength and skin texture parameters in separate steps. By linearising each step of the fitting process we derive closed-form solutions for the recovery of the respective parameters, leading to efficient fitting. The proposed optimisation process involves all the pixels of the input image, rather than randomly selected subsets, which enhances the accuracy of the fitting. It is referred to as Efficient Stepwise Optimisation (ESO). The proposed fitting strategy is evaluated using reconstruction error as a performance measure. In addition, we demonstrate its merits in the context of a 3D-assisted 2D face recognition system which detects landmarks automatically and extracts both holistic and local features using a 3DMM. This contrasts with most other methods which only report results that use manual face landmarking to initialise the fitting. Our method is tested on the public CMU-PIE and Multi-PIE face databases, as well as one internal database. The experimental results show that the face reconstruction using ESO is significantly faster, and its accuracy is at least as good as that achieved by the existing 3DMM fitting algorithms. A face recognition system integrating ESO to provide a pose and illumination invariant solution compares favourably with other state-of-the-art methods. In particular, it outperforms deep learning methods when tested on the Multi-PIE database.

Item Type: Article
Subjects : Electronic Engineering
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Hu, G
Chan, Chi
Date : 22 February 2017
DOI : 10.1016/j.patcog.2017.02.007
Copyright Disclaimer : © 2017. This manuscript version is made available under the CC-BY-NC-ND 4.0 license
Uncontrolled Keywords : face recognition; face reconstruction; 3D Morphable Model
Related URLs :
Depositing User : Symplectic Elements
Date Deposited : 24 Feb 2017 14:43
Last Modified : 16 Jan 2019 17:12

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800