The System for Reconstructing 3D Shape and Expression Deformation from a Single Face Image
| Shu-Fan Wang | Shang-Hong Lai |
| shufan(at)cs.nthu.edu.tw | lai(at)cs.nthu.edu.tw |
| NTHU | NTHU |
Project summary
Facial expression modeling is central to facial expression recognition and expression synthesis for facial animation. In this work, we propose a manifold-based 3D face reconstruction approach to estimating the 3D face model and the associated expression deformation from a single face image. In the training phase, we build a nonlinear 3D expression manifold from a large set of 3D facial expression models to represent the facial shape deformations due to facial expressions. Then a Gaussian mixture model in this manifold is learned to represent the distribution of expression deformation. By combining the merits of morphable neutral face model and the low-dimensional expression manifold, a novel algorithm is developed to reconstruct the 3D face geometry as well as the 3D shape deformation from a single face image with expression in an energy minimization framework. To construct the manifold for the facial expression deformations, we also propose a robust weighted feature map (RWF) based on the intrinsic geometry property of human faces for robust 3D non-rigid registration. Experimental results on CMU-PIE image database and FG-Net video database are shown to validate the effectiveness and accuracy of the proposed algorithm.
Related publications
-
Estimating 3D Face Model and Facial Deformation from a Single Image Based on Expression Manifold Optimization
Shu-Fan Wang and Shang-Hong Lai
In Proceedings of European Conference on Computer Vision (ECCV), 2008. -
Reconstructing 3D Face Model with Associated Expression Deformation from a Single Face Image via Constructing a Low-Dimensional Expression Deformation Manifold
Shu-Fan Wang and Shang-Hong Lai
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2011. (Accepted)
Last updated on April, 29, 2011.
