Manifold of Faces
The manifold of faces is a mathematical model of the set of all realistic-looking faces. We estimate this manifold from a few hundred high-resolution 3D face models, and impose a coordinate system on it so that every face has an "address" consisting of roughly 20 numbers. We can then compress and reconstruct faces with very high fidelity, and synthesize novel faces with very high realism. The image shows faces generated along a straight line on the manifold; underlined faces are real.
Background & Objective: Good models of human faces are necessary for animation, security (recognition/identification), and video coding applications. Previous attempts to model variability in human faces have depended on purely linear models. These can easily generate objects that are not face-like. We seek to estimate a compact function that efficiently codes (compresses) and decodes (reconstructs) realistic 3D graphical models of faces.
Technical Discussion: This project is a demonstration of charting - a new data reduction technology developed at MERL. Although it takes roughly one million numbers to specify a high-resolution 3D face model, it is very likely most of the visible differences between any two faces can be described with just a hundred numbers. In short, the set of all faces is a low-dimensional manifold that is embedded in a million-dimensional ambient space. We use charting to estimate the manifold from a small sample of faces. Charting constructs a low-dimensional coordinate system on the manifold, plus smooth functions that map between the ambient space and the new coordinate space. Operations that are difficult to do with raw face models become simple linear operations on the charted manifold: Â morphing (interpolation); caricature (extrapolation); comparison (distance).
Contact: Matthew Brand
Technology Area: Computer Vision
Modification Date: September 12, 2007