Voice Puppetry

    •  Brand, M., "Voice Puppetry", ACM SIGGRAPH, August 1999, pp. 21-28.
      BibTeX TR99-20 PDF
      • @inproceedings{Brand1999aug,
      • author = {Brand, M.},
      • title = {Voice Puppetry},
      • booktitle = {ACM SIGGRAPH},
      • year = 1999,
      • pages = {21--28},
      • month = aug,
      • isbn = {0-201-48560-5},
      • url = {}
      • }
  • MERL Contact:

We introduce a method for predicting a control signal from another related signal, and apply it to voice puppetry: Generating full facial animation from expressive information in an audio track. The voice puppet learns a facial control model from computer vision of real facial behavior, automatically incorporating vocal and facial dynamics such as co-articulation. Animation is produced by using audio to drive the model, which induces a probability distribution over the manifold of possible facial motions. We present a linear-time closed-form solution for the most probable trajectory over this manifold. The output is a series of facial control parameters, suitable for driving many different kinds of animation ranging from video-realistic image warps to 3D cartoon characters.


  • Related News & Events

    •  NEWS    ACM SIGGRAPH 1999: 4 publications by Hanspeter Pfister, Paul Beardsley, Ron Perry and Matthew Brand
      Date: August 8, 1999
      Where: ACM SIGGRAPH
      MERL Contact: Matthew Brand
      • The papers "Voice Puppetry" by Brand, M.E., "Feline: Fast Elliptical Lines for Anisotropic Texture Mapping" by McCormack, J., Perry, R.N., Farkas, K.I. and Jouppi, N.P., "The VolumePro Real-Time Ray-Casting System" by Pfister, H., Hardenbergh, J., Knittel, J., Lauer, H. and Seiler, L. and "Computer Vision for Computer Interaction" by Freeman, W.T., Beardsley, P.A., Kage, H., Tanaka, K., Kyuman, C. and Weissman, C. were presented at ACM SIGGRAPH.