3D Face Recognition
By applying 3D face models to robust automatic face recognition we are addressing the most critical factors limiting performance: illumination and pose variation. We have developed a novel system for capturing the 3D shape of a human face from a sequence of sparse 2D silhouettes from multiple cameras (or video) at affordable cost and with no manual user interaction. Using silhouettes decouples the geometric subtleties of the human face from the nuances of shading and texture. Our framework presents several computational and algorithmic advantages over the existing techniques for 3D face modeling. We are now applying our modeling framework to illumination- and pose-invariant 2D/3D face recognition with promising results.
Background & Objective: A "Morphable Model" is an analysis-by-synthesis framework for capturing 3D models have from 2D photograph(s). Models are fit by finding the shape/texture parameters which will render a synthetic 2D image which best matches the observed image. The key advantage of our silhouette-based approach is that it does not rely on dense image/texture correspondence in order to estimate our model's shape parameters. Instead, the face shape is estimated directly by way of its own intrinsic cues: the occluding contours (as represented by the object's silhouettes). The texture information, on the other hand, is simply lifted and post-processed after the shape estimation stage is completed.
Technical Discussion: We use a linear combination of "eigenheads" obtained by Principal Component Analysis (PCA) of a training set of laser-scanned 3D human faces. The PCA coefficients are used as model parameters. We establish correspondence between faces with an efficient error metric (boundary weighted XOR). Our parameter estimation uses a "downhill simplex method" (which requires no gradients) and can be readily adapted to existing graphics hardware for computational speedup. Moreover, the resulting parameter recovery is surprisingly robust with respect to partial and noisy input silhouettes (with both positive and negative clutter). Our overall model acquisition pipeline is considerably faster (x10) than existing state-of-the-art techniques which rely on dense correspondence (eg. optical flow in "Morphable Models") and is robust with respect to illumination and texture variation across the face and we have now achieved near automatic model-fitting using MERL's state-of-the-art face/feature detection technology.
Publications:
| Technical Reports: | |
| A Bilinear Illumination Model for Robust Face Recognition | |
| A Practical Face Relighting Method for Directional Lighting Normalization | |
Technology Areas:
Computer Vision
Graphics
Modification Date: July 3, 2007