TR99-20

Voice Puppetry


    •  Brand, M.E., "Voice Puppetry", ACM SIGGRAPH, ISBN: 0-201-48560-5, August 1999, pp. 21-28.
      BibTeX Download PDF
      • @inproceedings{Brand1999aug,
      • author = {Brand, M.E.},
      • title = {Voice Puppetry},
      • booktitle = {ACM SIGGRAPH},
      • year = 1999,
      • pages = {21--28},
      • month = aug,
      • isbn = {0-201-48560-5},
      • url = {http://www.merl.com/publications/TR99-20}
      • }
  • MERL Contact:
  • Research Area:

    Algorithms


We introduce a method for predicting a control signal from another related signal, and apply it to voice puppetry: Generating full facial animation from expressive information in an audio track. The voice puppet learns a facial control model from computer vision of real facial behavior, automatically incorporating vocal and facial dynamics such as co-articulation. Animation is produced by using audio to drive the model, which induces a probability distribution over the manifold of possible facial motions. We present a linear-time closed-form solution for the most probable trajectory over this manifold. The output is a series of facial control parameters, suitable for driving many different kinds of animation ranging from video-realistic image warps to 3D cartoon characters.