Happy and Sad Face Classifiers
The goal of this project is to have a computer determine if a detected face is happy, sad or neither. The algorithm works on either photographs or video. Each frontal face that is detected in an image is classified according to its facial expression. The same approach can be used to determine other characteristics of a person from his or her face such as age, gender or race as well as other facial expressions.
Background & Objective: The happy and sad classifiers use the same machine learning framework as the face detector to learn two classifiers for happy and sad faces. The resulting classifiers are very fast. One use for this technology is for marketing purposes, for example, to determine if a customer is enjoying an advertisement or store display. Similarly, age and gender classifiers could be very useful for gathering marketing data.
Technical Discussion: To classify a face as happy or sad, the fast face detector developed at MERL is first used to find all faces in an image. Then the face is normalized to fix its scale, rotation and position. This is done by finding nine facial feature points (such as the corners of the eyes, etc) just as in the face recognition technology developed at MERL. Once the face is normalized, the separate happy and sad classifiers are evaluated on the face image. If either classifier scores above threshold then the face is classified appropriately as happy or sad.
The challenge of learning a happy or sad classifier is to find a set of features that can be used to distinguish smiling (frowning) face images from other face images. This is done using a machine learning framework which is given examples of smiling (frowning) faces and examples of every other facial expression. The AdaBoost learning algorithm is used to select a set of simple rectangle features that best discriminate smiling (frowning) faces from all other facial expressions. If the classifiers are being run on a video sequence of a face, the accuracy is improved by integrating the classifiers scores over time.
Technology Areas:
Computer Vision
Off the Desktop Interaction and Display
Modification Date: June 16, 2006