Recognizing Talking Faces From Acoustic Doppler Reflections


Face recognition algorithms typically deal with the classification of static images of faces that are obtained using a camera. In this paper we propose a new sensing mechanism based on the Doppler effect to capture the patterns of motion of talking faces. We incident an ultrasonic tone on subjects' faces and capture the reflected signal. When the subject talks, different parts of their face move with different velocities in a characteristic manner. Each of these velocities imparts a different Doppler shift to the reflected ultrasonic signal. Thus, the set of frequencies in the reflected ultrasonic signal is characteristic of the subject. We show that even using a simple feature computation scheme to characterize the spectrum of the reflected signal, and a simple GMM based Bayesian classifier, we are able to recognize talkers with an accuracy of over 90%. Interestingly, we are also able to identify the gender of the talker with an accuracy of over 90%.


  • Related News & Events

    •  NEWS    FG 2008: 2 publications by MERL researchers and others
      Date: September 17, 2008
      Where: IEEE International Conference on Automatic Face and Gesture Recognition (FG)
      • The papers "Ambient Intelligence as the Bridge to the Future of Pervasive Computing" by Wren, C.R. and Ivanov, Y.A. and "Recognizing Talking Faces from Acoustic Doppler Reflections" by Kalgaonkar, K. and Raj, B. were presented at the IEEE International Conference on Automatic Face and Gesture Recognition (FG).